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INTRODUCTION 



CONFERENCE PROCEEDINGS HAVE BEEN AN ANNUtt FEATURE 
ctnpf 1Q58 PRIOR TO THAT DATE, AN ANNUAL REPORT 
WAS GIVEN ABOUT THE CONFERENCE AFTER ITS CONCLUSION. 

EACH PARTICIPANT THIS YEAR SUBMITTED HIS REMARKS ON 
SPECIALLY DESIGNED PAPER. EACH PAGE WAS PHOTOGRAPHED 
tn ORDER TO PREPARE A PLATE SO THAT THE PROCEEDINGS 
JouTbE PRINTED AS SUBMITTED BY THE AUTHORS, WITHOUT 

editing by the cta staff. 



fTA ASSISTANT RESEARCH EXECUTIVE, DONALD P. GLASER, 
CONFERENCE PROGRAM MANAGER AND JOSEPHINE BROCK IS 
CONFERENCE PROGRAM SECRETARY. 
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first general session 



SPEAKER: j. RICHARD HARSH 

SOUTHERN CALIFORNIA REPRESENTATIVE 
EDUCATIONAL TESTING SERVICE 



TOPIC: 



"BIOPSY OF AN EDUCATIONAL GROWTH 



BIOPSY OF. AN EDUCATIONAL GROWTH 



J. Richard Harsh 

Southern California Regional Office 
Educational Testing Service 



preamble i W h at Is The Entity.!. 

The anatomy of education has whi ° h iS 

being variously referred to as a Ration, an a ^ ocountabillty .,, T he fas- 

metabolism. This n f w 1 e T t ^ ty .^ a ® h ^! * heT are widely used by persons having 
cinating thing about labels is that theyjre wid y which the label 

Although some may deSedthat 
is a fetish for the moment, and it will . fe3S * onal journals and in- 
newspapers, textbooks, entire i s ^es p ® nominat ed this entity 
numerable regional and rational meeting in deed is this entity 

(accountability) for pr ime focus Is it a newly identi- 

that has been attached to the an y distress 9 Is it a mutational 

fied but always existent gastro-^testin^ distress . & boil or 

growth such as an extra eye, hea j the orga nism? The biopsy to 

rash? Or is it a new homeostatic P cal diagnosis, but it is 

be presented does not assure s ^screte cl b ^ avio ral manifestations, 

hoped that we may consider the etiology ox 

Old Wine Tn New Bottles 

There is substantial evidence in the recorded Idstoiv of 
previous ordered societies that t h e ^stract ^idea^ o accoun^ for 

recognised as a concern to the ^ f®r t he development of the young and the 
organization and action. ( Concern f o . . dentified in the tribal 

means by which such should be att^ed ® s b tha feudal societies or the 
societies, the GreekorRoman 01 00 lony of Massachusetts when the 

Deluder-Satan laws of the Hew England colony oi m waa 

nation was first organising a new as by the adults in 

assigned to and performed by work group ) t the society 

the family, a tutor and a student o g» should develop -and 

had expectancies for how the members of that society snoux 
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contribute. Moreover, individuals and groups were responsible for that 
development; and if a member of the society did not develop as expected, 
ere were even such strict account ability consequences as banishment of 
the individuals responsible from the society. The Deluder-Satan law 

i+^+fS C +Jf ltab ^ :Lty -. pr0n0Uncement ^ was 811 interesting one., for it explicitly 
stated the rationale and predicted consequences if the younger generation 
did not acquire the skills and knowledges from appropriate reading. In 
essence, that colonial society was saying that education of the young was 
the. imperative to prevent them all from falling into the tentacles of the 
devil. Parents, teachers and the town council were to be held accountable 
(an pay a fine) if the desired developments of the young were not achieved. 

flpr»r>nri+ ov>* i • + suc ^ ^f s i°fi ca i data is accepted, then one must conclude 

i i y is an abstract concept that has been and is present in any 
Society holds its members responsible and accountable for 
specified individual development. The major variations in this concept 
appear to be related, to the question, "Who is responsible for what?" This 
is particuiariy crucial as one views the institution or system of education 

roles (LHpfnn yea f S - The reorganization of institutional 

v. o res P° ns ibilities) has resulted in the system of public education 
being ambiguously and variously perceived and held responsible for a vast 
array of the observed developments of the population. Society grimaces 
from the pain of. such troublesome problems as defective communications, 
antagonistic social and cultural values, economic distress, changing 
vocational, demands, and the disappearance of previous roles of individual 
participation in the society. 

. There is no doubt that society is reacting to this dis- 
comfort by saying someone or something is responsible! A natural avenue 
to explore is the means by which the population has learned to be as they 
are. Ergo - the educational system could or should be responsible. More- 
over, education must be accountable for what it produces. 



a 



, , u -X x, s ? for the fi S ure of speech, "old wine in new bottles," I 
would submit ,.nat societal concern for its members’ development is "old 
wine, while "who is responsible for what?" is a current "new bottle" 
demand that there be accountability assignments to the institutions or 
systems of the society. The "new bottle" seems to be the label of 
accountabiiity for education. However, the prolific discourses on the 
label (accountability) suggest that this new bottle for old wine holds all 
sorts of elements, medicines and substances. 



What’s I n This Vessel Labelled Accountability ? 



. ^ -*- s apparent from the literature and ongoing discussions 
™ a ™ 0U ? ! ne !! lbers of our society (and our educational systems) believe 
the bottle labelled ’’accountability" may contain any of the following: 



1. A strong drug that will cure the educational organism of an 
acute illness. At this time the exact characteristics of the 
illness are not defined. 
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productivity of the system. 

to invisible gas that is odorous and unpleasant but will gradu y 
disappear in the atmosphere e 



a.** A .. 

- U2£ 3 S5S of e . 

5 - tod^pr^uctiw^st n dead e and n non-^oductive 1 cells e of 1 the^organism^ 

nf course there are a host of other concepts defined or 
im^ed.that t£i 

are convinced that their delirn expectancies for what will 

in store for them. The disparity o rnn tributes to the ambiva- 

result from the phenomenon of accountability contn 

an^confusion in emotional and cognitive reaction. 

The emergence of accountability has not been limited to the 

variety of expectations, it has a^een »s ^tS^edf^oUowing 

^Tms" S ttie^procedures^r 1 systems that have been suggested 
" t^" tt thtch accountability may be explicated and realised: 

i P P B S (Planning. Programming, Budgeting System) - a managemen 

S&iin: aa-s^-s prissy. «. 

SSM *£- X - 

states to design such a system for education. Indeed 8 “ 

accounting manual are being compiled and field-tested at 

time • 

2 * d^igne^to a asSst U school Ii personnel 1 'to <1 describe' n and 8 monitor 0 the 

operation of a program or system. 

3 - ssssf-sst 

components of a system and their interactions. 

4 . Performance Contracting - A school system ® ^contract 

srr^crsrWov?^ 

will produce the desired outcomes in the P^^^ractor will 

Guaranteed performance contracts specify that t 

no^be paid if the specified outcomes are not achieved. 

a Turn-keying - the process whereby a program established and 
5 ' operatS^effectively by a performance contract is adopted by a 
school system and operated by its personnel. 
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6. Program Auditing - the independent examination of an educational 

program or performance contract to verify results, processes, 
personnel and the contents of the activities. * 

7. Education Vouchers - suggests a means by which parents could buy 

™ a ? 6e market ' the particular school program they selected by 
presenting a voucher for tuition issued by a school district or 

age " c J.* The implementation of such a plan implies the 
availability of financial and program audits, standards of 

educational quality and evaluative data to potential parent- 
purchasers. ^ 

Incentive Pay - the rationale suggests that teachers will be paid 
on the basis of the performance or achievement of their pupils. 

. . , The Political arena has produced a wide varietv of studi es 

commissions and task forces to accelerate the design and implementation of 

acc^AhnT 1 Sy ^ 6mS Wh f ch Wil1 P rovide the information^Kta-base for 
modpT^ t bll ^ ty ’ ^ P omraissions directed to work on cost-effectiveness 

iUustr a f 1 ™ 3 n? I fv,r ti0 ! 1 ’ and . bud e etin « and accounting systems are 
illustrative of the apparent conviction that if a plan and procedure can 

bmtr^ ’/ leg i Slati If insure delignationTre^onsS 

S r nnst-affectiveness accounting for the educational 
program. The proliferation of these commissions and task forces has nro- 

Sgif " ^ est /^ o shall be the one to whom the educators L e 
ultimately accountable?" And the divergent ramifications go on! 

Problems, Chronic Pains. Symtom Complaints Associated 

With the New Entity 

that the total I!l!^ d ® r “ S ^ aternent of the year ®ig ht be the observation 
professional community related to education does not 

Sussed) 1 y “ aC ° 0UntabiUtjr < as divergently understood and 



to result from *1 enc e , ambivalence or negative reactions appear 

to result from imprecise answers to the following questions: ^ 



1 . 



2 . 



fye the schools to be accountable for? For all aspects of 
affective and copitive student accomplishment and development? 

of the”?^- S - th ?V he f e mUSt be ex P licit and detailed statements 
an ^ 1C1 f ated outcoines 111 behavioral terms that are sus- 
ceptible to observation or measurement. Most educators would 

f taile ? specifications of anticiS“tcomS do 
the eve ^ , those that do exist have not been accepted by 

neede^or'appr^^ate . SCh ° ol - related Potions as reLvantf 

Who shall be accountable? Cold logic would suggest that each 
person whose task it is to influence learning - teacher, principal 
curricuLum coordinator, counselor or whoever - should be held * 

a u?^u? ble _ f0 L ££ - ecisel y i!^t £art of the educatio nal outcome 
- b h — — ^£ fect - ^- rectl .y through his own efforts . Ambivalence 
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of environmental effects. . , 

3. How shall and'out- 

— ssr. ~ ^ 

outcomes to the various inp providing reliable in- 

ssr - ■■**— •' " 

educational program. . . a .. n+ 

Lm By whom shall accountability be de termined ? The insis enc dent- 
fe^aluation is biased thfother hand, 

external auditors or evalua . pp j susoicion about the 

program participants have recurrent y treatments that will 

relevance of the process, 

be used as a means of judging programs of evaluation, the 

SHSSSsr. 

evaluator. 

Although these four questions ar e c “^^^^pScit^swers) 

straightforward, they are ( f^.°“^^?have enduring acceptance and 
complex and time-consuming if y Richard E. Schutz in his 

application. This seems to be caps i ^ ca ii e d attention to the 

article Development i£ ( Qn H ” instruction" in education. He 

great distance between const ruc__ : . ressure "and potential are not 

observed that publicity, desire, po construction. It seems that efforts 
sufficient to convert constructs in ability through performance 

to this point in of behav^ral objectives 

contracts, PPBS or cntenon-ref construct-construction 

are still largely near the construct end of the construe. 

continuum. 

The incoa^lete a^ gener^^^^^f ^ deas 

accountability systems ® eem3t °J* e “ Cities. Many assume that rhetoric 
or constructs are transforme definitions, issues and implications 

and intellectual exercises Systems. In contrast, 

are sufficient to result in de JT . A ore a process than a point. To 
the transition from theory t ° P r ^ * P as accountability 

me, a salient question is: How do you get an ^ interactions ^ch are 

^ent ^ ^t^ P—^ 31 ° f a 

school system? 

Felix Lopes has made some useful obseCTa f f^Sfusts 
why efforts to implement accountability have been unpro uc 

^ '“^1^ and legislative mandates in time and scope, 
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2 ’ pr£dl^ staffr Untabmty ” easm ' es on '“'^><5 and uncom- 
3 ‘ ^Tmems? by ” anaseinent that accountability is an end rather 

4 ‘ Smdtofo? n th f SySt ®” Without build - u P °f background and under- 
°? J he accounting policies and procedures. (What infor- 
mation vail be used for what puipose.) 

5 * ^fi^? n ,° f , great accomplishment and parsimonious answers with 
over-simplified procedures and minimal resources. 

Low or unknown reliability and validity of the measures to obtain 
data on which the accounting of accomplishment is to be judged!^ 

p «££ pass £ 

t*“f^S documentation^ ev£u“ 

p anning of the educational program may be facilitated. 

What About the Measures of Change? 

sene Of our diStr6S5 •“« 

tolved ^bt“a“pSe°^.?|S°Lre e be 

Sd^ h s?^fdS m rSlti^T v J ». th t e ? f bebavi o rs of 

three, five or instructional periods such as 

thTn^r S b\ he Sl “ d " d oftoe meas^e b ^ce a sc^es 

been^ignored. rellable —»«« * » d other forms of error vari»ce ha“ 



change on vdiatevS^rseSted^te^Hte^^ the ne f sslty to sho “ 

Such a reaction ^n^fn t °? nt ® ts “ d answers to the test items." 
only represents Ts^^of f 

pa.’S’ssas riJyxH-Sslr"! 

reflected in th» “ brt *f feLal t usable and enduring achievement is 
reiiected in the subsequent year of instruction. 



in the accountabiUty ,ode the 

sampled by the measuring device ^ w^ie there is frequent in- 
educational program is a common J? . measure s should be used to observe 
sistence that nationally s . near iy equal reaction that such 

change in student performance, there_^^^J ^ md objectives of 

Roger Lennon has ably indicated J^^^^no^ye^altogether 

that can be said positively about ^ch an app ^ be translated into 

clear how a series of c ^ te ^f“^Sn or growth. Lennon also observes that 
units that will yield measures _g ^ will be time-con - 

criterion-referenced tests (of good quality for the develop- 

suming and costly to pro uc • ^g i ess well explicated than the 

ment of criterion-referenced mi 5 . t It seems clear that criterion- 
development of norm-ref erenced tests. I as „ e ll as the need for. 

rigorou^devSo^ent^^theory^methodoiogy, instrumentation and analysis. 
Since accountability is 

and valid information, the con ? e ™ f °y oblem .^rime limitations of this 
obtained is a critical profession P . mus t be increased study and 

brief biopsy pron*>t that have been dis- 

SSTS professiona^ccdleague^such as Lord, McNemar, Stahe, Wardrop, 
Lennon and Croribach. 

Treatment and P — — accountability 

It is not the intent of this^al^is gfs 

lament" or a ne« version of the Te ”;^® b ? lit as applied to the organisation 
my conviction that the idea o ac oo venerable and potentially powerful 

and conduct of an educational syst management and improvement of the 

approach to increasing the understanding, an a « tQ the me ans of 

educational program. This h c “^^af to be ^sliced, 
nurture that may allow such a potential 

Henry Dyer, who has devoted 

analysis and conceptu^sation of^he gpl the ma ny facets andmmplex 

educational systems, has * ntten planned for. Dyer suggests that 

th^thre^interrelated^spect^of^accountability are concerned with legal, 

moral and informational considerations. 

The legal aspect of accountability^ 
for ensuring that all the people ^^^Jhe focus of attention is upon 
accordance with the goals of voters policy makers, administrators, 

ssz: ssis^^ss’-TSa-'^ 
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the authority and contractual relationships by which the system operates. 
In the broad sense, all of the parties have a responsibility to some or 
several other parties. If the concept of legal accountability is to be 
a positive force in improving the quality of education, it is imperative 
that authority and contractual agreements are based upon sound educational 
goals and explicit agreements for the means of achieving the goals , 

The moral aspect of accountability has been defined by 
Dyer as "the ingrained habit of having care for others, particularly 
students, and being generally concerned for their growth and well being as 
persons." While moral agreements cannot be legislated into belief and 
personal action, there is needed concern that contracts emphasizing methods 
and outcomes do not overlook or inadvertently destroy the moral account- 
ability of persons engaged in the educational process. 

The information aspect of an accountability program seems to 
be most fundamental to make the educational system functional. It is im- 
portant to emphasize that an educational system is a people system. It is 
a system made up of all kinds of people - policy makers, parents, teachers, 
administrators, pupils - all of them accountable for continally making 
decisions about how well the system is serving the pupils 1 needs. As 
people are constantly making decisions, there is a need for everyone to 
have accurate and comprehensive information about the context, input, con- 
tent, process and outcome of the program. Indeed all of the several types 
of people related to the educational system must be held accountable for 
supplying, receiving and using information for decision making. Moreover, 
it is imperative that the exact nature of the information provided from 
parents, pupils, teachers or policy makers be available and understood by 
all if the multi— directional flow of interactions and decisions may con- 
tribute to an effective system. 

Dyer suggests that the three aspects of accountability are 
different ways of looking at the same unitary concept, and no one makes 
any educational sense unless the other two are also kept in mind. For 
example, everyone connected with an educational system has a legal and 
moral responsibility for providing necessary information to keep the system 
working effectively. At the same time, the information is necessary as a 
means of determining whether the people are fulfilling their legal and 
moral obligations to the goals of the system. Furthermore, there is ample 
evidence to suggest that legal directives are not likely to be accurately 
implemented and have much effect unless they are girded by moral commitment 

to the importance of meeting the educational needs of the children served 
by the system. 



The foregoing aspects of account ability which are operating 
in a people system must have clear definition and acceptance by ail 
participants. If there are some participants who believe that by imposed 
authority or external mandate they are being held accountable for un- 
acceptable outcomes or pupil achievements that are beyond their influence - 
then the resultant "hang— up" by those concerned is the redirection of energy 
to oppose or neutralize the "external enemy" that is threatening the 



internal ft**. This has been .referred to syndrome." 

T^ioa counts of bhoee Pe«ei™« ^ oblenls , they , create that 

What ??\£p riiffLStv " Of course the "they's" may be certain parti- 
:?p^tfL ae^eS Policy makers, or oust an undefined populate that 
inhibits the efforts of the "we" group. 

If accountability is to make a transition from concept to 

ship^but 1 artive^eii^^ thrreKted^icip^tHhat include 

s srss 

::sfsrr better 

results than those obtained with present practices and attitudes. 

In addition to building acceptance of the' “J^^stiSmce 
, . *i t ji*i *up necessary for each school system to find 

^££SSS£** ^^"r^thlt^l’and 

d ^ar en ^tl“ n be Sd.su-nl^d I*— d for the 

SEES must^e^oded 

o? this inadequacy has «M Stufflebeam ^to 

■^jg a.-iijg^sra . 

“S conditions, and output. Inp^^^the^^ ^ 

^S^outpu? is^concerned ^ith the attained accompUshment of the 
^t“tte J schor^^rnt “ay influence how tto ins tructional 

This model indicates the necessity for all ^f^^ b i® S ^Q d ^ e 
measured and appropriately interrelated in various combinations to produ 
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Figure 1 

The Pupil-Change Model of a School 
as a Social System 
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interpretable indices by whichthe staff-^erve ^U^own^efforts 
are producing desired pupal Such ^h 00 i effectiveness indices 

Sf coicepSi^ as being derived through a series of regression analyses, 



From this kaleidoscopic view of the entity labelled^ 

"accountability," we have viewed ^J^^^^^agnosis of the existing 
^SSonT/pro^osirfortTmure health of the anatomy of education 
has not been attempted. 

The analysis of the symptoms, . treatments^d growths^suggests 
that the construct of accountability h °^ initiated and implemented in 

^rfl^Te' compart -» an 
industrial operation dealing with inert materials. 

The statement by Dyer summarizes the foregoing conviction 
”It must be constantly kept in^d that the educational P-es^not on 
all fours with a manufacturing or mdu .^interacting with other human 

process in which human b iv°La sureable and predictable. Education 
beings in ways that are imperfe y living minds that are 

does’ not deal with inert integrity, and, 

instinctively concerned, first p , , . uri + h the world around 

second, with reaching a meaningful accommodation with “ pro duct 

them. The output of the educational a dva^fit i= an 

r SOTS, can fully under- 

stand or predict." 

I have enthusiastic commitment that we may make progress in 

.. ‘ 

environment through efiective . suege sted when we consider 

accountability. At the same tame, humility as suggesteu^ ^ 

the dynamic and unique complexl y behavior and existence that is not 

with systems for our 

"Sll syste^, v “As that we will also try and see the man or 
woman each individual is trying to be. 
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PERFORMANCE CONTRACTING: THE DALLAS EXPERIMENT 

Dr. Donald R. Waldrip 

(Dr. Donald R. Waldrip is Assistant Superintendent: 
Accountability and Personnel Development for the 
Dallas Texas, Public Schools. He received his 
B A and M.A. degrees from Midwestern University 
and* his Doctorate iron Nothem Colorado University.) 

Dr. Cowan, when inviting me to participate in this 

conference, asked me to give " s ? bs S ““®^ t 3 louwill have in hand in 
Hollywood last March, updated by the resu y wi comp u e d. In 

November." If “^tuati^’wasnotc^pleted until eight 

addition to “ dead iine for submitting this paper had past -- not 

or ten days after the deadii taam to perform — enough crises 

‘Ho* accountability and performance contracting 
have not been ^“^'^VggeSVin DaUas lately is^gregation. 

On August .8, 1971 the F ederal District Court gave i amendments, two 

Dallas to implement a Prescribed desegregation^,!^ very llttle m 

stays, and three clarifications la , clarifications. We opened 

Dallas but implementing orders, 8 5^®’ “ all available manpower has been 
school two weeks late, and since that *1^, hones constantly and 

riding buses chasing do™ fumoi ‘UaSSrf^ Sousand students 
S S a y kvI e ^ SS several weeks now been working 

literally 1°^ drive' for acceptability in Dallas^ 

brought about frustrations , but it f^deLg^gaSon will ultimately 
for desegregation. But perhaps ou the compensatory 

accomplish more performance contracting, combined, 

programs we !_ ^ since the passage of the Elementary 

and Secondary £Si Lt gave us the * 

sst^rr . is jsesuss 

call the "culturally disadvantaged. lo and clever ne „ software; 

l^ertS tasSS aldWnara for Mrlines- 

th^Hr of innovation as 

energetically as aj^one . ^ chough we got a considerable booat from 

Title I and other forms of federal aid, our costs went up. bripled 

“ Ihe lSt 10 years - mainly because of new construction, salary 

17 



O 

ERIC 



20 



■J 



o 

ERLC 



r s ^^r ial “^i: r r- 

« f °“” d - didn't have much tTbe^rouTof °" “* P “‘ ° f °“ r teachers - 
averaging only ^“year • f «hi™? ga^for^luU^owr 

iZkxr^tt* 0 irini: r ^--g-^ss.’S 0 ^ 

they had been in 1965. 8 ware worse o££ ln 1970 than 

dirty linen for^bSTv^ng 13 reluctant to «-»« out his 

embarrassment even no^as l L<* * certaln 8en8e of 

is the knowledge that ^irtual^ iLy t*™,™ 'TT t0 d ° 80 
country has had the same experience L m . w ^ school system in the 
dollars after Title I was «.«!T.e J? u ' M yaars and £lve billion 
the cycle of under-achievement that sees children°frm arned , hOW t0 break 

childre^to^school?' J ° bS ^ 

Changed in educmof 5S® 

£e aSSer b ted ^ "o 

teacher ratios, put a libratv , en ° Ugh T ey to lower P U P £1 - 

Sl^b ^:; ShSJTSk ^^^‘^“^Wonal 

a stingy public had never all“ed“ to ^ttern! “ nt Unchallen * ad 

of a chance as ^r^lfteve^ited^nSLos 011 ^ 6 C ° ** them! "° t 88 ” Mch 
spread Title I funds around so thinlv ttat fhl !°e o^ 001 systems 

any impact. Nevertheless, we were given a raLmnM “T® 7 C ° Uld not have 

r ^ tS a dld DOt . Ju3tl£y tie” ZLSrinTt^nlrSe^ S Se the 

pu c oes not oelieve it is getting its money's worth from public education 
frustration behind the drive 10^0^0^111^ Uf “ a ,P r ofessional 

for purposes of devising new educational strategy. Per-nulrtl 1 ^^?!?* 
do not really tell us what it costs to educate ^student^n S^^n'® 8 
is what it costs to keep a student seatid for a yeS. ’ y teU “ 8 

"leaming-unit" A J^t h Tw ral ® v f lt measure . Dr. Allen argues, would be a 
of total 8 building expense teacher ' 8 salary, portion 

required to moveY^S fr™ ™ materlals 

math to the next highest w^l % „ 2^. reading, writing, or 

school to another; they would be higher in a acZl°S I^jori^ of™ ” 
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rMldren from low-income, black or Spanish -speaking families than they 
JSfSe in a school with a majority of white children from upper-income 

h0meS ' Developing such a new accounting system would enable edu- 

cators to show the public how much learning «as p r °duced by a certain 
amount of investment. It would, moreover, enable educators to^hlft^ 
resources back and forth within a budget — testing, for example, 
vaTue of teacher-aides in one classroom against the value of educational 
technology^n another and of programmed texts in a third. 1" aach case 
would be related to output - and educators who prescribed various 
iifehipg strategies would be held accountable for the results they pro uce . 

Accountability is, in essence, a statement of policy. It 
states that educators will accept responsibility for their performance — 

per s onne 1 ^and * the^ub lie t S and* that 6 tha^ contr ac ^invo Ives Jhan showing 

as professionals, educators can overcome - or will learn to overcom 

ing an idea int ° e * f f^' contractlng ls one such technique. It is not 

the only one. Voucher plans are another — and so, indeed, is any sys em 
effort to relate educational effort to student achievement. 

effort to relate^eouc^ ^ ln Dallas led u3 t0 try performance 

contracting. Our interest in it led us to two distinct Programs -- one 
financed bv the Office of Economic Opportunity, and the other by Title i. 

I Xh today to describe the Title X program because we controlled it from 
Le start: chose the student population, outlined the P er f ‘anv 

wrote the request-for-proposals, defined the conditions under which • 
successful contractor would have to work, and negotiated the final contracts. 

The entire process has been carefully monitored by the mas t 
precise scientific methods. According to our most recent ^es, for 
example, every administrator involved has lost an average of 13.1 P * 

eiven up 46.3 percent of his weekends, and antagonized 75 percent o his 
Sves to the point where 100 percent of them threatened to go home to 
mother an average of 3.4 times. Nevertheless, innovation marches on i 

Dalla8 ‘ First, a note on the 0E0 program. It involved about 600 . 

students in grades one through three and seven through ^ne in tw ^® c 00 S ’ 
these students are matched with another 600 in a control group. The 
subject areas are reading and mathematics, both of \rtiich were su con rac 
bv 0E0 to Quality Education Development, Inc., of Washington, D.C. 

Contracts for two service components, audit and ^ “ 
explain these terms a little later on - were awarded by 0E0 to Batteiie 

Memorial Institute and Education Turnkey Systems , Inc. 

The two programs resemble each other in princip » 
course; the major distinction is that 0E0 designed one program, Dallas the 

19 



O 

ERIC 



22 



K1 „> 



other — hence I feel I can discuss the Dallas program with more authority. 

a .. First, the target group. Last May, we ran an analysis of 

underachieving high school students and selected a group whom, on Se bfsis 
of our experience, we believed were highly susceptible to dropping out. 
y August 31, the first day of school, our predictions were proven 

su^rf < ? accurat «: fully 50 percent did not show up. We divided the 
survivors into an experimental group of 960 and a control group of 700. 

The experimental group were all students in grades nine through twelve 
attending five Title X high schools. 

decided the program should concentrate on three kinds 
of instruction: first, basic skills - communication and mathematics- 

second, occupational skills: and third, achievement motivation - helnine 
youngsters develop a determination to succeed. 

f ,, The characteristics of both experimental and control groups 

3 n r : ^ ey W fo 4,8 standard scores below the national 50th 

percentile in reading, 6.2 scores below on vocabulary, and 4.9 scores below 

Sk * US ; Thelr teachers and counselors indicted Sat each 
seemed to lack any desire to succeed in school, or any realistic goals in 

.. . When we set up this new shop, then, we chose to go after the 

£25*2 C ™ ? e l S ' L ° n8 bef ° re We P icked them » however , we began thinking 
about the kind of program we would ask contractors to bid on. We started § 

STpLp™ 1 -? Md^b^k ]l 69 i* ^ a Plannin8 Advi sory Group that comprised 
P p and I think it important to describe this group • 

v. a. . 1116 me “bership included only five employees of the school 

The oJw C ? n f al u taff admini strators, a principal, and two teachers. 

£** “? 5 i ncl lf ded the president of the Classroom Teachers of Dallas, 
i ? ! loCal filiate; seven students and ex-students; one 

school board member; and the rest, residents of the target neighborhoods 
representatives of local colleges, local businessmen, and officials in 
Dallas civic agencies. 

The cynical way to view this is that we were trying to 
minimise opposition — and that, indeed, was one of the fringe benefits. 
Performance contracting seems by implication, at least, to impugn the 
competence of teachers, and one might expect their representatives to 
oppose it. But we are fortunate in Dallas to have NBA representatives who 

imDrwW 17 / 16 ^ t0 *£? interests of thelr members and to sound ideas for 
worth^M* UCat ^ on * They agreed that performance contracting was a concept 

toarw! £ ; Perhaps ’ the y felt > it might be a step toward training 

teachers to aim for performance. 

Yet minimizing opposition was not our principal objective 
in expanding the membership of the Planning Advisory Group. It is difficult 
for any educator to admit that laymen might know a thing or two about 
educating, but we decided to investigate the possibility. 

„ . , . ^ humility paid off. Among many other benefits, it led 

us to include a somewhat offbeat course in the occupational training 

lZ?ZlKr, r T eSt - fm ^T aaU -- draftl ”8 for girls. That suggestion 
came from the businessmen on the advisory group — and every girl enrolled 

in the drafting program has already been spoken for by a local industry. 
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request to eUgible t£ *c otSraSts loosed by the 

2S£Ll££S.r « e t pre-bid^conference in May, and chose the 

successful contractors in July. . RFP? 

Now — what had we asked for in the tu?r. . 

The tot" list of performance criteria and conditions is 

such too exhaus^to^epeathete " 

those which, X ^ 

have to gain 1.4 ^-leveUin galnlng . Payment to 

0.5 grade-levels CMs partic P P stud ent gains; unless every 

s^ra^en 1 lA-y£ gain, the contractor would not be able to recoup 

his costs. SeC ond, in achievement motivation, ^^""^^succelsfil 

to reduce dropout rates below those • • • ^ the Unite< i States. The 
Title VIII dropout prevention projects th^gh^ attendance in the ac hieve- 

retention rates, however, wo contractor would have to do to main- 

^szsjsr^r 

define performance cri *T*f of occupational training is employability - 

components. The essentia ^ rt „ d itions as well as by educational 

but this is affected by economic editions as veil y 

excellence. However, we did 5 he . 2 5 local companies to work with 

S-2JS fZZEZ&SXS £ ^ » « &lso 

in judging the quaXi^ . subsidiary ofthe Meredith 

Corporation, w ° n ^ be ‘°"^ a 3 fM aS lmt motivation aAd occupational 
Stag^wralso requested proposals for two other components; audit 
and management l8 intended to keep everyWy h^est - 

to prevent a repetition of the - f “^es^Sten to ^eSm^i and 
wanted an outside agency to approvethetestsgivento xp design 

control group students ^roughout the pr ogram 8lngly 

so that we could appraise the effectiveness otvarr and> flna ix y , to 

and in combination; to ascertain ® could be properly compensated, 

certify the result. . 

contracted for on a penaity-incentive basis^the ““ lt ° r 

fixed fee. We chose Educational Testing Service co provxu 



Management support, as the name implies, is to help out 
management -- in this case, the Dallas school system. Performance contract- 
ing was new to our staff; all of them have full-time duties, and we did not 
want to divert then, to an unfamiliar job. Hence we contracted witt tte 
ouncil of Great City Schools — again for a fixed fee — to provide a 

supplemental, staff that would act as liaison between the school system, the 
contractors, and the auditor. y * 

In addition, the Council of Great City Schools felt that 
placing a few of its representatives on our staff temporarily would 
increase their expertise in performance contracting. In a sense, even 
though their people have significant experience in this area, they would be 

serving an internship - learning along with us so that they could Zev 
help other school systems. 

u . ^ last aspect of performance contracting that I feel you 
shouid know about is the "turnkey” aspect. The three instructional cLpon- 

thP ro f t.r r / r °? ram “*\ math > conmunica tions, occupational training - employed 
the contractors methods and materials, but they employed Dallas personnel/ 

oroerL bl / ln ^°!S RFP * Moreover > we insisted that the contractors' 

f b so 1 de f 1 f xed * hat tfte y eculd be adopted throughout the school 
system if we elected to do so. 

Mn . , 5S “ t ia "turnkey” means. Thus performance contracting 

e viewed not only as a tool for improving student achievement, but as 
a tool for improving the effectiveness of teachers. Each contractor agreed 
iLZft m t f acher " ^ his methoda if those methods work. Each aLo® 
th l US real ex P ense fi^es, so that we could appraise 

a nro«t" ff ? iVe a eS ^ ° f hiS program * We expected each of them to make 
W \ Sign f d the contracts, and if they could deliver, we didn't 

eaC ^° f made ‘ But we did w^t to be able to compare 
^£ir learning-unit costs against ours, so we could decide whether their 
methods could be extended to other students within our budget restrictions. 

In connection with the "turnkey” aspect of the proeram T 
venture the opinion that performance contracting poses no threat to any 

^nstitutiona iCt T? Th? C v e i S * BUt lt: d ° eS P ° Se a threat to teacher-training 
can SST' u lf T ? iok ° 1 ? r New Centur y or Jim-Dandy Educational Systems 
can teach teachers to teach potential dropouts to read, after all the 

nrpdir? Ph * D * f. in OUr diversities have so resoundingly failed — then I 
predict, we will see a lot of Ph.D.'s out of work during the next decade. 

Mp do WiT .. a f t ls l°° early our experiment to judge all the results. 

h l °? r 5 8 population had a much higher attendance record 

- h ^rob!bl ^° 1S; 5?f Se y0U ? gsters whom we identified as probable dropouts 
previous^years/ "" showed U P more often than they had in P 

hi oh T ^ e e yf lua tion of the 0E0 project in elementary and junior 

high reading and mathematics has not been made available to the par ticioa tine 
school districts^ at this date. Indications are, however — based on the 
resuits of interim performance objective tests - that students averaged more 

that indi^ ?, 8r ^ Wth during one year of instruction. These results^uggest 
11Z f systems of instruction below the high school level, when 
! PP ° d ted by sot J nd » alm °st continuous monitoring and staff development, stand 
g chance of succeeding with educationally disadvantaged students. It is 
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difficult at this point to generalise 

m ost observers agree “ tB £ t S"££l£ £ouf ^obfi-s^est for learning 
enthusiasm among their students . bringl g discovered negative 

among the participants. Some tfbSreen scores in criterion 

correlations among disadvantage tests — indicating that the 

referenced tests and scores in Stan influencing results, that the 

“«“iLi b rrferfnfe"i e tems e :r 8 e too easy,- that any number of additional 
factors could beMamed Slina- 

^ctr ™ did 

the control group, even though the comp ^ £* gt of all wor lds. The 

gains. An economist might consi , it did no t reach its guarantees; 

si ^ i£icant » ^rStfa! design e ^ e Ae^h school reading and mathematics 

program called for all students fr gr^ps!^' ^mentioned 
randomly assigned to experimen a an £or scboo i during the first 

earlier, many of these students did n ^ t virtuously impossible, 

three or four weeks; consequently, ^ndom^assignment w^ int( / a treat - 

As the less interested ^Ity in aptitude between the treatment and 

ment group, leaving a gr P we r e from the "target population, 

control groups, even though S P co - V aried out in all analyses — 

The bias resulting from this < dii 5pa *\J. because of the difficulty of totally 

an almost totally unsatisfactory , intelligence. The average 

adjusting for mathematical 1 ^ “ *Ue the 

intelligence of the treatment groupfor intelligence of the 

average for the control group was Se.ll.The^average^^ ^ g^ control 

treatment group ^dicatt that although the research design 

group was 82.16. These tigur excluded from the 

called for students with IQ s of less tnan /u students enter the 

“ h r p»b?r^de^: °— 

hardware, - d “s^rtri/th^m^ric^ 

gain based on the results of pre-and-post standardxzed tests^The con^ ^ 

group, although averaging 11 points n for che treatment group, it 

ir^greare^ to be deemld significant at the .05 level of confidence; 
is not great enuugu w l-f nMum r e among groups did not ditrer 

therefore, the conclusion was that performnce among gr p 

more than what would be expected by chance. But again cne r 
bounded by i. 

SthMgLr ^ r p^ n &f (“Sh and Leffth graders performed 
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better than ninth and tenth graders; (3) the more a student was absent, the 
poorer he did in the program. These three conclusions are not very startling 
but they do provide corroboration for some of our biases. * 

Students in the reading program gained an average of .53 years, 
while their counterparts in the control group gained .48 years. As with the 
mathematics students, this conclusion persists in spite of the fact that the 
average IQ of the students in the treatment group was 10 points less than 
the average of the students in the control group. But again, the differences 
in achievement were not greater than would be expected by chance. 

The finding of no significant difference is difficult to 
ftP °f t * le 334 participants in the reading program scored better 

than 75/o on thexr criterion referenced tests. The correlation between scores 
on the criterion referenced tests and scores on the post standardized tests 

W f S /*r 5 V n0t 3 Very significant correlation, but one which indicates that 
students who tended to do better on the criterion referenced test tended to 
do poorer on the standardized test. 

In addition to the questions concerning difficulty of criterion 
items and cultural biases of standardized items, perhaps the most relev an t- 
question we could ask would be ”how do we get disadvantaged students to do 
their best on post tests?" One student whose scores indicated an eight year 
loss in reading ability — a preposterous conclusion — had actually answered 
more than 75% of his criterion test items correctly. Three more whose scores 
indicated a loss of six years also answered more than 75% of the criterion 
test items correctly. In fact, 79 students showing a loss of 1.0 years or 
more in reading fell into the same category of 75% or better on the interim 
per ormance objective tests. As in the mathematics program, the brighter 
students in reading performed better; the eleventh and twelfth graders 
performed better than the ninth and tenth graders; and the more the students 
were absent, the poorer they did in the program. One additional finding 

re a es to the reading portion of the experiment only: females performed 

better than males. 

Reading and mathematics scores were Increased if the students 
were also assigned to achievement motivation classes. Attendance in these 
ciasses, a segment of the contracted program with Thiokol Chemical Corporation, 
definitely contributed to greater reading achievement. Students in achieve- 
ment motivation classes attended school -- their regular classes, not iust 

^/^« en L m0 ^ iV£C1 °^ < \ la3SeS -* m ° f the « marked Improvement 

over the 73% attendance of the target population the previous year. Results 

of deportment scale evaluations of regular teachers would indicate that the 
attitudes of these students toward school was superior to the control stu- 
dents, even though the measured aptitudes of the control students were higher 
than those of treatment students. 

Vocational training under Thiokol was in three areas: auto 

mechanics, machine metals, and drafting. 30% of all students in the three 
courses reached the graduate level of training and were placed onto jobs. 

okv reac ? 6 j apprentice level. 23% achieved the level of assistant, and 
25% reached the helper level. Since all of these achievement levels are 
varying levels of employment, 95% of the students enrolled into these 
specialized, individualized programs achieved an exit level of employment. 
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Inasmuch as all reading and mathematics participants did 
not make positive gains, the confutation ^age- 

confounded. This computation, h al£ students making a negative 

ment support group students for which both pre-and- 

gain made zero gain. I made some kind of gain and 181 did not. 

K.'TS ™ K »'*•”» 

’■ 1.*S J3K STS MK. - 

, j «. „n Tlno cost of this program was §44Z.Jo per scuueuw 

131 made no gain at all. The cost ot P g decrease if m0 re students 

could^e'^moved fr^^he ^^^gg^^^^^a^DaXias^inas^c^as 

"Here is where X amount of your dollars went, and nere x 

difference which that i-^rod^ ^ . which 

to carry out controlled experiment. t«ts. We 

encouraged s ” e ^ a ^ y i ^® r '®| U ^splayed by primary and junior high 
are encouraged by the ieteres P ^couraged by the effect the achievement 
students in our OEO project. a „v« 4 moment We are very encouraged 

motivation classes had on attendance ^chievemenu We « y 
by the performance of our vocational students. 95/. ot tnem ux 

what it has takenjwo 7®“^“ really°eval™ting performance contracting. 

Rather, we w er e^cvaluating^cert^n^instructional^systOTS,^and^some^of^the^ 

the^condLng S year , an'd hopefully w ^^“L^grLiesVs^er 1 U?OOo‘prtLry 

a^-ssr»: wx: -- — - 

1971-72 school^year.^ ^ reguUs o£ our £lrst year experience offer some 
in that special we owe it to the students, for they 

fe^-difad^ab pl^moft “f ^hopIs^oTthe” 

careers, this seems to me a fair trade. 
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TEACHING PERFORMANCE TESTS 
AT THREE LEVELS OF ACCOUNTABILITY 



W. James Popham 

(W, James Popham is Professor at the Graduate School 
of Education, University of California, Los Angeles. 

He received his A.B. degree from the University of 
Portland, as well as his M.A. also, and his Doctorate 
degree from Indiana University.) 

Most current conceptions of educational accountability xecog- 
nize in one form or another, the educator's responsibility *° P*°“°** 
worthwhile changes in the skills, knowledge, »nd attttud« of learners. 
And in recent months there has been ample rhetoric persuading educa 
tors that an accountable stance is* indeed, meriton • * 

E Morions* inexpensive. So, until we devise some Practical pro- 
cedures for promoting workable systems of educational accountability, 
this phenomenon, as so many other educational breakthroughs, will 
remain an improvement vehicle at only the verbal level. 

In the following remarks an effort will be made to supply more 
tangible suggestions for implementing accountability systems ^y first 
describing the teaching performance test, a specific measurem 
tactic which can be employed in various accountability approaches, 

Jext by distinguishing between three different forms of educational 
accountability^nd, finally, by illustrating how teaching p^formance 
tests can be employed in each of these three accountability systems. 

Xhe Teaching Performance Test 

perhans the most perplexing problem facing educational re- 

searchers and eJSlu^rs P siSce the turn of the century has been how 
*o measure a teacher's instructional skill. The most widely used 
t o measure^ a tea classroom observations, and pupil performance 

on standardized tests, have all proved dismally inadequate.* There 
were* two major defects in their approaches. First, they w ®*® M 

too process -focused; they tried to isolate ''goodteachingt^hniques 
even though subsequent research strongly suggests there are few, if 
any" pedagogical ploys which will invariable work in the 
strictional settings teachers encounter. Second, xf not process 

*A morVext ens ive analysis of the defects in such “J* 8 "*® 8 ’ /5JuS*in 
ariuantaaes of the teaching performance test approach, can bs 
Pooham J "Performance Tests of Teaching Proficiency! Rationale, 
Development^ * and Validation. " AafilisaiL Besgft l ff il 

volume 8, number 1, January 19»i. 
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sed these mea sur em e n1: techniques failed to take into account the 
fact that different teachers pursue markedly different goals and, 

° f teachers with differing instructional 
phases yielded little $ 0 re than confusion. 

To eliminate some of these difficulties, a previously untried 

thelca®?? teChn i qUe 1 . haS been tally tested since 1965, namely, 

the teaghina Effformhnce test, while there is still considerable 

W ° rk t0 * be carried ont regarding various aspects of teacb- 
o SLTrr; they have been employed in a sufficient number 

further ^ifira?i^n“ " lth reS " ltS " hlch appear to warrant their 



First, what is a teaching performance, test? >Such tests nmuiiio 
an estimate of a teacher's ability to product a presp^ifi^ b Jnvior 
hange in a group of appropriate learners. Here's how it works. 



£i£s£, a teacher is given an explicit instructional 
objective along with sample measurement items showing 
how the objective's achievement will be measured. 

|_ econd, the teacher is given time to plan a lesson 
designed to achieve the objective. 



Third, the teacher instructs a group of pupils for 
a specified period of time, perhaps as few as a half 
dozen students or as many as a whole class. 



Fourth , the pupils are measured with a posttest based 

J? e but unseen Previously by the teacher. 

Pupil attitudes toward the instruction are also 
measured. These measures of pupil cognitive and 
affective results serve a? an index of the teacher's 
effectiveness. 



a ! e ^° r “ an f e t6StS haVe been used successfully for periods as 
ort as 15 minutes or as long as ten hours. One of the questions 

period be to**** r frf 6arCh a ? alysis is how lon 9 must the instructional 

skill Jimilsr!3 +i, a re ^ labl ! estiro ? te of the teacher's instructional 
iSJf: sin JJ®rly, there is interest in many of the shorter, more 

n^i y . USable * P ^ f ° rnancetests en ^P loye d to accurately predict a 
teacher s success in longer, more realistic teaching situations. 



tests is n n TO©? ra +h©! h K sub ^ ec ? matter employed for such performance 
tests is novel, thereby reducing the likelihood of the learner's Dre- 

vious familiarity with the topic. Becuase the same instrSc?Snal 

objective is ^ployed for all teachers completing a given performance 

i i-i? •<> compare different teachers with respect to 

r ln accon Pj lshi n9 such preset objectives. This, of course, 

iLk i± L an9le * By boldin 9 constant the instructional 

W m«ier the «sk. C ° ntraSt the ablUty of *“««■* teachers 



r 



_ . •_ __ n+rn I other relevant conditions so 

that all^teachers sane opportunity 

2-SSS for in * 

equities in disparate learners' entry behavior. 

As indicated ^ wen^no^ th^°teaching 0 p«f ornance 

educational aeon ™*J^J^ S t^^ v „ent.’that is, to help teachers 

get J^ST UsJ». that 

teaching performance tests can £ or partTSTlarly 
is, to discover which teachers a pa later paragraphs some 

bad at this_type ^ ^^"'““parformaice tests can be used for both 
STSeTe purposes^under various accountability systems. 

Before leaving teaching ^ rf °^"^ e hich S qe h ier”ly ’ aris^ when 
points must be made to assuage the <to*^ »hich generally 

educators consider this .ensure. SLSKto me^ed, there are typ- 
Beyond the P roce<tora ^"® S ^® n ^irvalidity of this strategy. Note 
ically concerns regarding * h ® only one competency of a 

that teaching performance t^ 8 achievc pre -specified objectives. 

teacher, namely, his abil ty should be involved in evalua* 

There are surely other dimensions^ which d ShouWbe ^ „ 

ting a teacher. Performanc one believes the mission of 

is an important criterion. for the better, then any indicator of 

:-2S£ ^fl^d^ consideration. 

some will say -But this ^^^what^eaching iS s really Uhe. 

Teaching for a full ^ y . , J i ^ nt3o _ lv ass igned learners for a 30 minute 
erent than teaching eight ra " d ^? f J | s But is there any reason 
lesson." Of course there are differences, bu Qn ghoxt 

to believe that % te *f 'nfsb^lnly blossom with pedagogical splendor 
term performance tests wil y kind of prediction involving 

in a regular teaching situation? y Teaching performance tests, 

human behavior is made ' prtbjjUUti-l ily. to sharpe „ our 

even in their current unsophisticaxea ioi«, y 
predictions. 



Educational Accountability Systems 

"" N 



NOW turning to a consideration of e^ca^onal^countabiaa . 

systems, let* s examinfe; lihd key j?5 : f eren + methods of implementing 
although we shall examine “f f s ' “ b asic P similarity in 

systems of educational accounta y * desianing or implement- 

all of them, namely, an e Sponsible for thl results those 

ing educational systems must bee ®®* rn ^£° This responsibility is not 

TslZl^ is - ponsibXe but> rather> 
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accountability oriertted educator to produce evidence 

coi. a ^ce h :/sir?« t ^«oS»r been produced in 

S in 0 tit^* 4 * 0 ””"^* P ^ ei ®d“®®* 0 " a ®*ands^r 2 dy to *ec^t*ositive^wr a ^ S ’ 
native Judgments of his instructional efforts an7«e su^^ueSt 
actions which may stem from thosp ▼ - * n ® su »sequent 

ar-jBttivwi £S~ SSswM- 

^j^ saaaraaa%sBB&“ 5^ 

bilitv ^ e »^ ief * dlffe i ence ln each <* these three forms of accounts- 

i 2® nature of the decision maker who demands evidenirin 
regard to the outcomes of instruction t ce xn 

°"*° r “f e colleagues to participate in is tJe ^ 

S55S52 

educational accountabi lity . on^else neid 9 b2 9 iwolvSl r in 0 f^h ,S " !a 
are TOt"cLrici^ reflardin 9 instructional modifications 

Professional accountability, on th* nth 4. - . 

ffi:SSa«&TrsS 

for^of 'accountability^bu^the « ^SfiSSaS® 

emerges from within the teaching profession. rather than i2- ll 

sasss? sssslfsigs? 



*A more detailed treatment of these accountaMi^t., < . . 



belng made each month, such evidence to be reviewed by specially dee- 
ianated teach^rl t€pt696ntln9 the subject field involved# 
would be made available only among the faculty, with thc P^ B ?^ h ^^ d 
Dose being to identify ineffective instructors so they could be helped 
to improve. This would be an instance of professional educational 

accountability. 

Public ac countability , as might be inferred froa its naae, ^ 

occurs when” widence regarding the quality of le " n * r *“* i “"T* ^ 
demanded by the public. For instance, if the school boa rd re iulrss^ 

that reasonably interpretable evidence of , n re "ai 

plied to school district taxpayers each year so that: **£•" ”* eB 

results-based judgments regarding the school system s effectivene , 
this would be a clear instance of pubUe educational accountability. 

In examining these three avenues to educational accountability, 
we see Sa^e eSSaon dimension in each i. a commitment on the part 
of ?hl educator to assume responsibility for the results of instruc- 
tion. Evidence regarding the iqwet of instruction on learners is 
systematically accumulated and then made available to one or more a - 
i«o« U the initiator of the evidence-appraisal scheme is oneself, 

then personal aceountabiUty is involved* if | h ® *"*** 4 ?°?| init- 
nniiMouBs then professional accountability is present, if the init 
iators^re lay*citirens or their representatives, then public account- 

ability is involved. 

Now each of these three forms of educational ac ^"l tab i} i * y 
carries with it the potential to improve the quality of education. 
However, each of the three forms contains acme inherent »trengthe and 
weaknesses which we should recognize. Those who mindlessly toss thes 
three variations into the same aceountabiUty bag really contuse the 
picture. Accordingly, we shall examine some pros and cons of each of 

the three alternatives. 

Turning first to personal accountability, how might it work? 
well clearly the focus in this approach is self-improvement. A 
teach er wants to get better at what he’s doing and becomes personally 
liable for the changes he is producing, or not producing, in his 
students Thus, for example, we might find a teacher who prior to a 
•instructional sequence routinely establishes expected levels of leaner 
oSrf^cy! STc^pafihS tSe.«tual levels of learner progress after 

instruction with the hoped-for progress, the ^fLtive^s 
defensible judgment regarding his instructional effectiveness. It 
mus^be stressed that a personal system of educational ^countability 
involves more than a casual "look-see" at the close of a " lns *™* tional 
scauence even if other teachers participate in the look . Most 
teach ers’ typically get a somewhat ill-defined impression of whether 
they" have been teaching well. A true system of personaleducational 
accountability involves the establishment of a systemat^ mechan ^ — 
whereby the instructor can appraise the quality of his efforts in 
terms of evidence regarding learner growth. 
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The fact that in a personal educational accountability scheme 
the teacher is accountable to himself is at the same time the most 
prominent strength and the greatest weakness. Since the teacher is 
the exclusive initiator of the system and, if he wishes, evidence of 
pupil performance is examined by no one but the teacher, the aoproach 
is obviously not very threatening. As a consequence, a good many 
teachers should be willing to employ it. On the other hand, since 
personal accountability schemes are essentially volitional and often 
private, it may develop that the instructors who most need to improve 
terns elves will be the least likely to engage in such approaches. For 
the weak teacher, even a private mirroring of his mediocrity is not a 
pleasant experience. . 

Now let's turn to professional educational accountability, for 
while this less private form of accountability fails to capitalize on 
the chief strength of a personal accountability system, it does cor- 
rect a major weakness of the personal approach to educational account- 
ability. The distinguishing feature of a professional accountability 
system is that it. is initiated by a group of professionals, such as 
a teachers' organization. Assuming it has the power to implement its 
plan, if the professional group requires teacher participation in the 
enterprise, then teachers will have to participate whether they wish 
to or not. This departure from the completely volitional nature of 
personal accountability obviously makes professional accountability a 
more threatening concept for many educators. On the positive side, 
the possibility that incompetent teachers could avoid participating in 
the system is eliminated by requiring their involvements 

In examining the merits of professional accountability we see 
immediately that since it is an intraprofessional operation it would 
be less threatening to many teachers, even if required of all. This 
is its major strength and represents the reason why a number of educ- 
ators believe professional organizations may lead the way in promoting 
educational accountability systems. On the negative side, it must be 
recognized that professional associations have historically been very 
reluctant to expose or expell their ineffective members. Thus, the 
possibility exists that even if the professional group is unable to 
improve certain of its members' skills, they may not be willing to 
make the hard decision to re-assign or even release a weak member. 

Turning to public accountability we find that citizens may act, 
typically through their elected representatives such as school boards 
or legislators, to require a system of educational accountability. 

!*?* ®xample, suppose a state legislature enacts a law requiring all 
district's in the state to choose one of several alternative schemes 
(devised by the legislature) for annually releasing to the public 
evidence regarding the results of instruction. Such a system is a 
c *f®5 ins J* n ce of a public accountability scheme, the effectiveness of 
which would depend on the quality of the procedures worked out by the 
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sL;srs :Ksss”r.s^r,£s l ss:ss.s , .s - 

Sws: asar'.-irsa'ss'JSsKS. 

they have basic doubts resardirgthewisdomofsuchapp 
will resist because of personalfears.forsurelyther 
terrified by a full blown accountability sys tea is the^p ^ ^ 

able to promote demonstrable growth in . aproun t ability sys- 

difficult to imagine poorly conceptualized public accountabi y y 

tsms^operated b/ill-informed or ^ ^^^^L^fu fccou^t- 

Sone people, fearing such misuses of a system, may resist an 

ability schemes. 

Using Performance Tests at Three Account ability Levels. 

Now quite obviously one can employ all sorts 
+ + * _ fi es h out any accountability strategy. This P P 

onlj constder how teaching performance tests might prove useful. 

Tn the case of personal educational accountability we can see 
w / J j mioht use a performance test chiefly for instructional 

how a teacher mig himself or with invited colleagues, we can 

improvement. Jither by himself or with in dlffere|)t eroups of ehil- 

2T2rAfS3STJSfi-» in 9 .«cc t sive efforts to improve 
her skUl -ap-ti^la^perfo.manc^test, 

^uldT^vised.- If a 

noted for subsequent use# , . . contrast her performance on 

were available, a teacher might wish to contrast ner perioroan 

tha^sh^raScsHat the^^O^th^percentile.^ertaps^sh^will be sangui^ 

teaching perforLnce tests will be directed toward instructional im- 
provement. 

Ror professional accountability systems, however, teaching 
performance tests can be employed for both instructio^ l improvemen t 

^oiu^isr 

1 

decisions# 

But more importantly, perhaps, is the P^bilitythat teachers* 
organizations will seize upon the use of teaching performance tests as 
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a skill assessment device to accomplish what they have always sought- 
cgntrol over entry into the profession. Consistent with a general 
thrust for professional responsibility, teachers » organizations might 
set up procedures, typically involving state teacher credentialling 
whlc ? individuals aspiring to be teachers would be 
obliged, alona with other factors, to display at least a minimum level 
or skill on teaching performance tests. 



Paced with external scrutiny and the possibility of imposed 
accountability schemes, the astute leaders of teachers* associations 
may also see the possibility of employing teaching performance tests 

foT a11 of thelr members. These teachers 
who displayed little ability to master such teaching performance 

tests could be given special instructional assistance via a service 
provided by their own teachers* organization. Such collegial improve- 
ment schemes might be seen as a means of warding off externally 
imposed teacher appraisal activities. Thus, for the professional form 

•° n ! 1 a f?° un *»bility we readily see uses of performance tests 
for both instructional improvement and skill assessment. 



.... Finally, for public accountability systems it appears that the 
skill assessment use of performance tests will predominate. In an 
effort to create school systems which can accomplish results, it is 
certainly plausible that the public might demand that teachers display 
at least minimal proficiency on teaching performance tests # 

One obvious form this skill assessment approach might take 
stems from the fact that ift many subject fields there are currently 
more applicants for teaching positions than there are jobs. Admin- 
istrators must choose among these applicants and, for the good of the 
children, will wish to select the most competent teachers from the 
many applicants# As an aid to selecting the very best applicants# 
the school district could set up a series of teaching performance 
tests which must be completed by all district job applicants. The 
amount of time needed from applicants might be three or four hours. 

In many districts this type of applicant screening examination would 
be similar to the procedure whereby applicants for a given graduate 
school must complete, at their personal expense, the Graduate Record 
Examination whose results are considered along with other information 
in deciding whether to admit the applicant to graduate school. And 
since enrichment topics could be used for the performance test sub- 
ject matter, any pupils from the district who participated in this 
applicant screening would be gaining new knowledge. 

Reflecting the fact that such schemes have now moved well 
beyond the theoretical stage, one firm. Instructional Appraisal 
Services*, now offers school districts a complete service for the 



♦Additional information can be obtained from Instructional Appraisal 
Services, 105 Christopher Circle, Ithaca, New York 14850. 
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Zlll lessons (2) the main cost of the testing is borne by the job 
nen urant (3) the appraisal service supplies complete supervision 

of ihe testing program pta” at its conclusion, a ranking of teacher 
applicants^according So’thelr skill in pronoting different types of 
cognitive objectives as well as positive learner affect. 

There are, of course, other ways in which teaching performance 
can be employed in public accountability systems. Indeed, the 
imaginative educator will find numerous uses for these tests, both in 
connection with instructional improvement and skill assessment. 

The reason that the teaching performance test 
ho particularly useful to proponents of educational accountability 
sys?«s Sl^coMpleie congruence with the central assunption of all 
accountability systems - a focus on the outcomes of instruction. 
Prorfirtablv there will be procedural problems in employing such 
performance' tests^to promote different forms ofeduc^tional account- 
ability. But, as suggested at the outset of these remarks, it is 
only when we descend from the ethereal plateaus of rhetoric to the 
practical world of public schools that such problems can re y 

faced and solved. 
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SYMPOSIUM I 

developing behavioral objectives 



Chairman: 



Dr. Jack M. Thompson 

Director of Instruction and Guidance 

Sonoma County Schools 



Participants: Donald Kase, Consultant 

Larkspur 

Roy Carson, Mathematics Consultant 
Sonoma County Schools 
Santa Rosa 
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PROBLEMS OF IMPLEMENTING THE WRITING OF BEHAVIORAL OBJECTIVES 

Roy M. Carson, Consultant 
Sonoma County Office of Education 



Much enthusiasm has been generated among administrators to imple- 
ment the writing of behavioral objectives at the teacher level, as a savior 

for determining whether a student has learned a specific -^"d^hrast "that 
t-TT-iat- f nr sending these messages down to teachers is a coined phrase, 
bfl 972 all «ache« will be Required to write behavioral objectives due to 

the state implementing P.P.B.S. 

Many consultants began retraining for the task to conduct inservice 
training programs for teachers centered around writing behavioral 
^he major problems that existed in conducting these programs were as follows. 

1. Teachers feel that this is just another burden handed down from the 
administrators. 

2. Another task to incorporate into an already crowded day. 

3. As teachers, they were not involved in the decision-making process. 

4. If administrators knew what was going on in the classroom, they could 
see teachers were already writing goals and objectives through their 

daily lesson plans. 

In honoring the request of district administrators, I conducted 
several workshops on behavioral objectives with teachers. At rs 
was a degree of nonsuccess due to the lack of communication between teachers 
and administrators and the level of the presentation. (Many teachers had no 
background on the definition of behavioral objectives.) Other W °^°P S 
involved the participants through audio-visual materials and a writing 
exercise; again a degree of nonsuccess. Success did occur when teachers 
were taken at their level of sophistication of knowledge on behavioral ob- 
jectives (Mainly working from their lesson plans and patiently bringing them 

along) . 

Stress was given to observing the behavior of students and writing 
objectives based on this behavior, rather than plugging students into a 
predetermined objective unrelated to the students in their c1 ® 88 * 8 

this process teachers agreed that trying to state objectives in terms of 
behavior is an eye-opener for planning. 
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SYMPOSIUM II 



Chairman: 



Participants : 
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PROGRAM PLANNING IMPACT 



Dr. George M. Kibby 

Superintendent of Schools 

San Dieguito Union High School District 

Cardiff 



Stephenson R. Parker 
Director, Instruction Center 
California Teachers Association 

J. Russell Kent 

County Superintendent of Schools 
County Government Center 
Redwood City 
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PROGRAM PLANNING IMPACT 



ACCOUNTABILITY 
(teacher's viewpoint) 



Stephenson R. Parker 



Stephenson R. Parker is Consultant to the CTA 
Curriculum and Instruction Committee and the 
CTA Instruction Center, Burlingame, California. 
He received both his A,B. and Ed. D. degrees 
from the University of California at Berkeley 
and his M,A. from San Francisco State College. 



Teachers are told that schools must be accountable to the 
taxpaying public for the funds they spend, and accountability must be 
Measured in terms of an improved product. In the broadest sense, then, 
accountability from the teacher's viewpoint is that teachers be 

accountable for the performance of their students only if ey 
the decision-making process which has a bearing on student performance. 

Teachers want to be and must be accountable. However, 
accountability is a two-way street. If teachers are to be accountable, 
thev must also be provided with the educational tools, parental, adminis- 
trative and political support needed to make accountability more than just 
a catchword. P If not, teachers will be forced to use their power to assure 
that they c^n provide the "professional" education being demanded of them. 

For example, are teachers to be held accountable and hence, 
responsible for overcrowded classrooms in the traditional sense? Is x 
their fault, or better still, should teachers be ^eld accountable for the 
inadequacies of the curriculum, textbooks, supplies, libraries, and 
counseling services, of which they played no part in determining? Is 
their fault that in many school districts buildings are being condemned and 

pupils forced to double up? 

Professional behcvior . The notion of professional behavior 
in education is not new; it is the nearness to it that is. What s 
professionalism? Simply, a professional professes to kn ^ better than his 
clients what ails them. The professional lays claim to the “ gllt 

to practice that profession as a vocation — the basis of license. The 
professional is expected to think objectively and inquiringly ab ° ub ™£ terS 
which may be for the layman .object to an orthodoxy and sentiment which 
limits self-inflicted painful intellectual exploration. 
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The Professional must demonstrate these four attributes: 

U; a high degree of generalized and systematic knowledge, (2) primary 
orientation to community interest rather than to individualized self- 
interest, (3) a high degree of self-controlled behavior through codes of 
ethics, internalized in the process of work socialization and through 
voluntary association organized and operated by the work specialists them- 
selves, and (4) a system of rewards (monetary and honorary) that is 
primarily a set of symbols of work achievement, and thus ends in themselves 
as opposed to some end of individual self-interest. 

These four essential attributes (the sociology of a profes- 
sional accountability) define a scale of professionalism, and a way of 
measuring the extent of accountability. 
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THE IMPACT OF PROGRAM PLANNING. 



J. Russell Kent 



(J. Russell Kent is San Mateo County Superintendent 
of Schools and a member of the California Commission 
on School District Budgeting and Accounting. He 
received his B.A. , M.A. and Doctorate degrees 
from Stanford University.) 



Do we really want to plan? Do we really want to clarify 
problems and their full implications ? Do we really want to do this in 
the full glare of the public aren a? It has always been assumed that 
planning was good, and in public institutions that there should be, at 
the least, full public knowledge of the plans, if not wide public p 
ticipation in their formulation. Do we really believe this . 



What are the hazards in public anticipation? What, for 
example, might have happened if the people of California could have 
known in 1950 that in the decade to. follow California s public school 
enrollment would more than double, requiring a trebling of the invest- 
ment in school facilities, while at the same time the operating cost 
per child would more than double? Would foreknowledge have led to 
more effective response than that which we actually achieved. Or 
would the public have backed away? 

In part, the answer to these troublesome questions lies in 
our answer to two other questions: 

(1) How extensive is our ability to foresee, and therefore, 
plan, not only for specific needs and problems, but also in the complex 
and dynamic milieu in which these problems or needs occur? 

(2) Assuming some ability to foresee with accuracy, can we 
communicate effectively to the public the highly complex nature of the 
interrelationships which are involved, in contrast to simplistic over- 

s implif ications ? 
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It is in the seeking of affirmative answers to these questions 
that organized planning systems such as PPBS, or Planning, Programming, 
Budgeting Systems, have begun to emerge, it is important to keep in mind 
that at this point in time such systems rest upon the hypothesis that the 

answers to the questions are in the affirmative — a hypothesis yet to be 
confirmed . 



Systems represent an orderly or sequential approach to the 
accomplishment of tasks, or to the description of events, which involve 
multiple and interrelated components . Whether devised deliberately or 
evolved empirically a system is directive, insofar as it presents a 
structure for activity. Since a certain rigidity is an inherent quality 
of any system” , it becomes critically important to examine fully any 

system which is to operate in the area of human affairs or in human in- 
stitutions . 



A critical examination of program planning systems, as they 
might apply in California's schools leads me to the following observations: 

(1) We do not yet have a working model of a PPB system which 
may operate in a school system. We have a concept, and we have ex- 
perimental development reaching towards its realization. 

(2) The techniques of systems analysis from which PPBS dev- 
eloped work much better in some situations than in others . They work 
best where there is a single goal, where this goal is tangible and subject 
to precise measurement, and when the outcome is proximate in time, i.e. 

when there is a minimum of time lapse between the input of effort and the ' 
output of product. 

(3) The nature of public schools which must deal with public . 
policy and operate in the realm of public affairs characteristically is 
opposite to the conditions cited above. School goals are multiple , not 
singular, presenting a strong probability of internal inconsistency which 
is unresolved in the goal structure. School goals tend to be controversial 
rather than accepted and understood. 

School goals tend to be intangible rather than tangible. They 
are never physical products which can be weighed and counted. School 
goals are realizable in a_remote rather than a proximate time period. 
Primarily we look forward to future performance of pupils rather than to 
occurrences which are to take place immediately. 

All difficulties notwithstanding, change is the hallmark of our 
age. Are we to participate in the management of change, or are we merely 



to respond to Its vicissitudes? Is there any real alternative to (tanning 
as "a way of life In our times? And If we are to plan In an open soolety 
I* £11 ° a „ we proceed but publicly, and how else are we to . deal with 
dte highly complex matrix of variables but to proceed systematically? 



SYMPOSIUM III 



effects of accountability on evaluation and research 



Chairman: 



Dr. Joseph T. Hanson 
Professor of Education 
Ambassador College 
Pasadena 



Participants : 



Harry Saallenburg, Director 
Division of Research and Pupil 
Personnel Services 
Los Angeles County Superintendent 



of Schools 



William tt. Stegeman 

Assistant Superintendent 
Curriculum Services Division 
San Diego City Schools 



EFFECTS OF ACCOUNTAdJ LITY 014 EVALUATION A! 10 ROSE ARCH 



Harry 0. SmallenbuAQ 



[Ha/tAij W. StiKilCcnbuAg is Director of the division o<5 1 

and Fupit PcMoimcZ Services in the Of free of the Lc >4 
County Superintendent of School*. He received lus b. A. degree 
from the University of Buffalo, and both •Us M.A. and Ed.P. 
degrees from the University of Southern California 1 



lit his keynote address to the [970 conference of the 
National Association of Pupil Personnel Adiwiistrators rn 
Indianapolis, Or. Leon Lessinger, former Associate United Spates 
Cotmtissioiter of Education, made the following statements: 

"Accountability has to do with honoring promises. It is the 
matching of intent to results; the comparison of what was supposed 
to happen to what actually happened. In education , accountabil y 
is the policy of demanding regular independent reports of promised 
student accomplishment for dollars provided. It is the liair-shirt 
policy— the responses to budget-passing time to t lie request for 
more money with the question , ""hat did you do with that other 
money?" It is not performance contracting , or behavioral objectives , 
or PPBS, though these inventions may be useful in implementing an 
accountability policy . In the final analysis , accountability is the 
final analysis— the hearing to get the facts, to determine worth, to 
check results. It brings to school instruction the same flavor of 
inspection and feedback brought by the fiscal auditor to school 

finance." 

This concern for accountability has occn nvuiiscsted 
in many programs and projects in California; notably, (PACl) *ojc 
to Advance Creativity In Education, (PEP) Preparation of Educational 
Planners, the Program Planning, budgeting Systerns, and Performance 
Contracting. The most recent illustration of this concer t ^or 
sustematie evaluation of services is the piibliwUon of the I 
California State department of Education cntUCcu Accountability in 
Pupil Personnel Services A Process Cuiiie ^or tae l cvccopmatt of 
Objectives. 1 ' 
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OtetectuM HcaiuA.cs 
Pretest JcasitA.cs 
En Route JcasuA.cs 
Posttest JeasuA.es 

Components of the Guidance Process 
Process Variables 
Situational Variables 
Pupil Variables 
Human Support Variables 
Cost Effectiveness 
detween-Pnognam Comparisons 



2) 



3 ) 



4 ) 

5 ) 



Superintendent of Schools, and’ others^ 6 ^ L Anflc£c * Cocott ^ 

an ^ ww->lc <Satt4($ffuiq progress can be made as 

-£ §£b 

?CMfc6y M 4^ i jJ! 14eltl ’ iCe edueat “ n 6 M ‘“‘n^tia&ue.iupviviio/ur, and 

J| definition of nonmat growth 
2) Reliance on interpolated nonms 

l\ *"* wtoA-io*™ equivalence 

4) Reliability of difference scones 
5 1 Regression toward the mean 

» . . . Recognizing these problems of admitiistnation and at 

c aiua^ion and measurement, research workers can continue to imrvtnuo 

m q mUiy a K UaMh by obwwi* Ac lo^^Sci^S 

raarfe to pupjl personnel administrators by Vr. Less ingen: 

. . . [I ”£lp them personnel diagnose and describe the dearee at nno- 

3 ) design precise, individual systems cavable at idontiini*,* ;« 

£7«1 4 SSi?'T ;c «» " 
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5) Provide plant &o>i involving and in^ofunbig otheA education 

ivonkoM and the conmuiittj about tuccettjul^ pAacticet . . 

Anvli/ the Aecognition that all school peAtonnel thane netp on 
tibitUij vSthtfic home and the client ^ achieving , eac/i 

pM,tneA. J being accountable io> \ executing thote p hue* in Much he 

m ott competent. 



55 




accountability and the failure syndrome 

by 

William H. Stegeman 
Assistant Superintendent 
San Diego City Schools 



Public and professional demands for accountability have resulted in a 
flurry of "new programs." The first evaluations of these P ro 8 r *®J * re *** 
annearintt The first round with accountability shows we are failing. I 
Jtoossme that .inspite of publicity to the contrary, ve are still trying 
to offer the same educational programs that brought about the demand for 
accountability? Are recent evaluations Just a better Job of revea ng our 

failures? 

The widely heralded "new" reeding program* gave great hope to 
dlaadventeged youth and parent*. "At laet," they thought, w® » . , 

learn to read." The results are .bowing the same dismal remit, es before. 
The truth la new program, have continued to teach reading In the same 
SatiSltUiutS SI?*®, before. Hew goal., new obj.ctlve. new technique,., 
new madia new learning styles, new programs have been neglected. Instead 

we have ignored research, used the same old I 1 ® 8 ®} 8 * 1 

objectives, over publicised our sketchy plans, faked action, and then are 

surprised by the results of the evaluation. 

The diversity of society and the adaptive needs of 1 * 

fresh look at what being educated means. If we observe the cold war 1 
decades of the 50* s and 60* s we see some disturbing tre^s. For exampie, 
the trend toward uniformity in curricula, overemphasis on ® c ® d *"4* - 

credentials (trowing bureaucracies as schools become larger, isolation of 
«„d«ci ind faculty from the "real world." and «®y ®tb« *lgn. of rigidity 
make educational institutions less and less a reflection of reality. 

We can not let theae trend, continue. We muat recreate a more ^lv *rae 
and re.pon.lve educational system. We muat expand ourtoucept.ofwh.tle 
the role of the etudent, what 1. school, what 1. learning, and what la 
teacher. We must develop many alternative path, to learning. In °ther 
“‘dst we must become accountable for what education acco.pll.he. end for 
what it 8 true purpose is. 

Evaluation must be less concerned with the needs of our educational 
*J25X »’« concerned with learning a. .een by the .tudent and 
society. The widespread assumption that the responsibility of ™ 
educational system is to provide opportunities for the conforming y 
successful students must be replaced. The new system " u,t pro ^J d ®. f 1 
exciting and useful education for every student at every step of the 
educational process. The failure syndrcme revealed by evaluation 
statistics is due to the vast range of potentially productive students 
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who remain outside the traditional and artificial succeaa n m (r. „ f 
evaluaclon'proflll^* ° Ut,lder * “* produeln * • negatively weighted 

o bJ ecr:/^\^i de r«:ju“t“„ y . be uMfui •• - ~ 

2vent h of -? e J its P ace ha8 been accelerated by the 

advent of student protest and demands of minority groups. 

••• possible? Wh ° Uy nCW technol °8 le8 ®ake a new structure 



• • • 



learner and "° Unt ° f te * e * rch * v * u « bl * on the 

There la a mounting conviction on the part of staff, students 

U dSJS.12 IT T titUtim ’ «“ «!•»•»* ti our needs. 

Itself ^ l lhll 4 l8 i . the f roce8S rather than the Institution 
itself that Is of greatest Importance. 

•'co.t A e«feM^!n°!J' ,t S b J Uty ,nd evnlnntlon has emerged the concept of 

Is UmltiTeo rt * .y n,0 “ UMtel y £ar too often the discussion of cost 
w *J d 1 th *I ue8tlon of expenditure reduction. Cost effective™*** 
must be less concerned with lines of a budget and more with th* m«! 

£ucr‘c? ”“?• ? f “ ,ff ““ ttmTC Si. to 

technical problems ‘r^ ^ . 

program, that cost effectiveness ££££ 

where making progress less costly for students Is^he 

learninn*.. **“ her,e 1 ««turlng actually produces’as much 

«-%-*£^rSwtrS«. lrKiependenC r “ dtn «* « whether 

of the knowledITL™J,Ii^ J } P « *??*“* 8r “ p better th * •ignlftcanc. 

the spsaUtyVle^li, “ d 

organization or content. P 7 cnanges in program, procedures. 
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SYMPOSIUM IV 



LEGISLATIVE DEMAND FOR ACCOUNTABILITY 



Chairman: 


Dr. Thomas A. Shellhammer 
Deputy Superintendent for Programs 
State Department of Education 


Participants : 


Clinton Jordan 

Principal Administrative Analyst 
Legislative Budget Coinnittee 
Sacramento 



Harvey Hunt 

Legislative Coordinator 
State Department of Education 



accountability or leadership? 



Clinton M. Jordan 



(Clinton M. Jordan is Principal Administrative 
Analyst with the Joint Legislative Budget Com- 
mittee. He received his AB and MBA degrees 
from Stanford University.) 

The term accountability has an array of definitions from 
the narrowest interpretation of a strict accounting of the 1C " 

school administrators spend funds to the brMdest interpretatio f 
. testina devised to determine now eTTective 

educators y ha™ S b een S in*teaching children. It is therefore difficult to 
discuss accountability without first establishing who is going to be 
accountable tS whin for what. However, this is not a serious prob- 
Iot if accountable activities are clearly defined. For example, state 
ipaislation requires the Department of Education to employ cost ef fee 
ti veness 1 measured? n an annual evaluation of al 1 categorical aid edu- 
cational projects for purposes of determining which school distric 
projects should be expanded, modified or replaced to produce the 
highest degree of program achievement and cost effectiveness. 

The Joint Legislative Budget Committee recently com- 
pleted an assignment to conduct an independent fiscal review * 

analysis of six major categorical aid education programs in Cali forma. 
a hacir conclusion of our two year study was that the data base essen 
tial to cost-effective evaluation of the programs under review di no 
exist, norhadplans been developed by the department to insure this 
data base for future evaluations. Thus the concept of accountabil 1 y 
becomes less important than a commitment by educational leaders to 
accept 5 thenar dstl c ks of accountability ~ measurable perfomance 
obiectives and evaluation systems. In other words, the emphasis must 
chanae from detailing "input" quantities to analyzing output measure- 
Sente iT. t™ progress of the student while he is in schoo and how 
well he is able to take his place in society when he has completed his 

schooling. 

Yardsticks for accountability can be imposed from outside 
the educational system or by aggressive leadership from within the 
system. For example, we believe that if the Department of Edu< cation is 
tn fulfill its leadership role and meet the increasing demands for 
accountability atthesiate level, it will be necessary for the department 



2 . 



with the aid of the educational profession to develop a comorehensivp 
evaluation system along these suggested lines: 

1. Develop statewide measurable performance objectives for 

f rogra !! , J 0 r service that receives state 
support. (Districts would be encouraged to develop ad- 
ditional measurable performance objectives based upon 

cc^aKi l°? al and ° fte ? unique needs, goals and resources.) 
Establish an evaluation plan for each statewide measur- 
able performance objective. This plan should specify: 

a. Data to be collected, e.g., student profile data, 
comparison group data and types and timing of pre- 
and post-tests in basic skills and areas other than 
basic skills, including criteria referenced tests 
and diagnostic tests. 

* 1 1 ■ use of data including how the data 

will be processed, evaluated and compared on a 
statewide basis. 

3. Rank and compare districts annually in terms of expendi- 
tures, salient characteristics and achievement on per- 
formance objectives as measured by statewide tests; 

4. Publish and disseminate annually all of the above 
information in the form of administrative guidelines, 
standardized evaluation report forms and statewide 
evaluation analyses. 

5 * h?UhocJ I e r 1e ^ of th ° se districts which ranked 

highest in the statewide evaluation and those which 

l°west. Disseminate information on the most 
?rr? C J 1Ve e ducational techniques observed and provide 
effective 6 t0 ^ 0Se districts found to be least 

IKP ha ma faars hav ? been expressed by educators concerning the 

rather than ?he' <t«!w an aT ,Ua ? 1o ?, wst< ™ ‘ r e ">P hasis *>" the dollar 

ratner than the student, dehumanization, overreliance on test result* 

emphasis on short-term educational performance, oversimplification of*the 

teKSTTL" - IS'lSf/S? ° f 8 System ° f tewarS? and p™?shSenS 

used. Neverthefess , d^isiois^^s^^^HfLXdfzed ZuZTl ^tlona'l 

decisfons'made they be the r,9ht °" as tha " 
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ACCOUNTABILITY: THE SEARCH FOR 

UNDERSTANDING IN PUBLIC EDUCATION 



Harvey K. Hunt 

(Harvey K. Hunt is Legislative Coordinator of 
the State Department of Education. He received 
his B.A. degree from Whittier College and was 
a CORO Foundation Fellow in Public Affairs.) 

There exists today a crisis in confidence in our public schools 

which if not met head-on immediately will seriously threaten the ?*® tem f ° r 
years to come. The signs of concern have been with us for some time. 

. The general resistance of the electorate to approve override and 

. Th^unwill ingneB s at the state level to restructure our outmoded 

. Srv«Jety C orJo £ piur“llter.ture which criticizes end downgrade. 

. The Introduction ^'panaceas, each of which la sold as the simple 
an 8 we r to all of our problems 

Unfortunately, there is no simple response to the critics 
which would restore confidence In the system, since (1) there are no universal 
rt^s^d wrongs and (2) the expectations for and the role of public educa- 

tlon Is constantly In a st^te of flux# 

How then can we meet the challenges presented to us b, the 
loss of confidence when we are dealing with a pluralistic system where our 
role and responsibilities are constantly changing? _ 

He must turn to the sources of criticism and ««fuUyexamlne 
the concerns which are expressed. A Gallup f^gg e should be 

^c^:r ^^ e of 

at any time through the use of reliable indicators. 

Consequently then to increase the accountability of the system 
is to increase public understanding of the facts indicating the * nd 

iea^esses of o« educational process. In effect this means to completely 
open the schools to scrutiny and criticism and to be willing o P 
sponsible suggestions when made. There is a substantial dange 
approachwhich most professional educators would be quick to point out. If 
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we simply throw open our books to public view the untrained eye may be lead 
to al l sorts of erroneous conclusions. The problem Is aptly sumnarlzed by 
Dewey In Die Rib 11c and Its Problem when he stateu that 

,, " If one wishes to realise the distance which may lie between 
facts and the meaning of facts, 1 let one go to the field of 
social discussion. Many persons seem to suppose that facts carry 
their meaning along with themselves on their face. Accumulate 
enough of them and their Interpretation stares out at you. The 
development of physical science Is thought to confirm the Idea, 

But the power ° f physical facts to coerce belief does not reside 
In the bare phenomena. It proceeds from method, from the tech- 
n que o research and calculation. No one Is ever forced by 
JHSi the collection of facts to accept a particular theory of 
£ helr m ganlnft so long as one retains Intact some other doctrine 
bjr which he can marshal them ." (Emphasis added.) 

fk. T ^ e P roblem of accountability that we are faced with today Is 

sublet !lt U J th tHe ab f en ? e of facts and data for public to view Z the 
? e overabundance of Information. What Is really required to 
n able Is not the production of miles of computer printouts or re- 

caSne l?ZZl ^ 4 ^ du8t ? “ the leade « b iP^ educators In comnunl- 

U ? ln l aymfln s terns what a11 of this Information means In 
V;"* ° f the education of their chlldren-where we are doing a good job and 
where we are doing a poor job and how we Intend to Improve our shortcomings. 

. i , , We ne f d to a PP roach this on two broad fronts. First, school 

^ "“I* ?? g 8Cally ^ cc °untable. We hear all too often that the 

C ° Uld 0666 a11 ° f the P ublic ' s expectations with existing resources 

«ih«£i°S. e J “ ana Sf Ben , t techniques were U8 ed, yet school needs always out- 
.. funds. Fiscal accountability means educators must be prepared to 

Identify priority problems with the direct Involvement of the people served. 

P £ obably ®° re d i«icult, aspect of this Is perform ance 
a ccountability . We must have a way of Indicating, In a manner the public 
can understand and accept, the quality of our educational programs. The only 
comprehensive system of measuring performance Is the statewide testing pro- 
gram which has only served as a format for confusion. Very few communities 
escape those newspaper comparisons of district's scores, state scores and 

national norms~all of which seem to Indicate that for whatever the reason— 
Johnny can't read. 

r „ . Educators must be prepared to provide the Interpretation of 

the facts that Dewey talks about rather than allow this directionless con- 
fuslon to continue. Further, If the testing system Is as many contend, ln- 
sufflclent. It Is the professional's role not to simply reject It out of 
hand but to build on It from other sources to recommend Improvements and 
most importantly, translate its results into understandable terms# $ 

. if Yf do not accept this challenge, the resistance to support- 

ing our schools will grow, along with more prescriptive mandates from the 

th ?J e 4 dea ii n8 ^ifb cla8S ®i* e » administrator/ teacher ratios 
and use of funds. It Is, therefore, our responsibility to get busy and exert 

leadership which the public expects so that the desire for accountability 
will be satisfied through better understanding. 
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SYMPOSIUM V 



ACCOUNTABILITY IN MATHEMATICS 



Chairman: 



Dr. Garford Gordon 



Research Executive 

California Teachers Association 



Participants : 



lego Kansen 

loordinator and Component Director 
lathematics Systems Laboratory Project 
Pomando Valley State College 



Edward G. Begle 
Professor of Education 
Stanford University 
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accountability in mathematics laboratories 

Viggo P. Hansen 
Professor of Education 
San Fernando Valley State College 



One of the current fads in mathematics education is the math 
tab. However the term math lab defies description. Schools that have 
these labs'generally developed them on the supposition that mathematics 
learning will increase along with student attitudes toward the subject. 
Theories underlying math labs are difficult to articulate and under- 
stand. Furthermore they are seldom supported by any substantive research 
Also there has been little teacher training and/or preparation on how to 



use a lab and evaluate its effectiveness. 

This paper is devoted to discussing the results found in a 
math lab program developed at San Fernando Valley State College in con 



junction with Los Angeles Unified School District. This program was 
begun under the auspices of Trainers of Teacher Trainers in 1969. The 



objectives for the math lab were: 



LONG RANGE OBJECTIVES: 

1. to develop a multi-component mathematics laboratory 
for training future mathematics teachers. 

2. to explore new teaching/leaming strategies xn mathe- 
matics education through a laboratory environment. 



to evaluate the effectiveness of the various laboratory 
components relative tot (1) master teachers, (2) student 
teachers , (3) Junior High students, and (4) college 
personnel . 

to create a computer monitoring program for scheduling 
students to various laboratory activities depending on 
4-heir needs. 



5. to evaluate the effects the laboratory has on the San 
Fernando Valley community and the Los Angeles Public 
School system. 
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SPECIFIC SHORT RANGE OBJECTIVES 



1 . 



2 . 



4. 



5. 



6 . 



to develop and define the mathematics laboratory as a 
physical plant. 

to schedule all Junior High- mathematics classes in the 
laboratory for orientation sessions, about 1500 students. 

to schedule selected Junior High mathematics classes 
for intensive instruction on a regular basis in the 
laboratory. 

to select and train master teachers from Patrick Henry 
to use the laboratory facilities for their classes. 

to select and schedule S.F.V.S.C. student teachers to 
teach their classes in the laboratory. 

to develop curriculum materials compatable with both 
the existing mathematics program and a laboratory 
approach to teaching mathematics. This will include 
preparing laboratory lessons, unit plans, and activity 
kits . 



7. 



to evaluate the effectiveness of the. various components 
and equipment in the mathematics laboratory. Specifically 
to evaluate: 



a. 

b. 

c. 

d. 

e. 

f. 

g. 



Laboratory Activity Sheets 

Computer Hardware 

Selected Mathematics Readings 

Filmstrips, Film Loops and Transparencies 

Audio Tapes and Listening Posts 

Student Projects - Individual and Group 

Mathematics Gaines 



8 . 



to train and evaluate teaching strategies in the mathe- 
matics laboratory. Specifically to train teachers in: 



a. 

b. 

c. 



Team Teaching - teams to be composed of college 
professors, master teachers, student teachers, 
and Junior High students. 

Team Learning — students working as teams 
using various laboratory facilities. 

Large Group Instruction using auditorium facil- 
ities. 



9. 



to use the time-share computer facilities to help master 
and student teachers in record keeping and to prescribe 
learning activities. 
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^ „, ve all future mathematics teacher instruction on 
* a° !aboratory f approach while enrolled in methods courses 
at San Fernando Valley State College. 

11. to establish an Advisory Committee of community, college, 
and public school personnel . 

a few of the findings that will be reported on at this 
session include: 

1. seventh graders gained 2.5 years in 

2. Student teachers who taught in the £ 

------ — 

discussion session, etc. 

i Master teachers reactions tended to be favorable, how- 
^er they have many specific concerns regarding the 

SSunt oJ time students should spend in a laboratory, 
role of textbooks, etc. 

4. Student attitudes as measured by a modlf ied Heyt- 
MacEachern Attitude Inventory indicated a loss or 
interest in mathematics for many students as a result 
of the laboratory experience. 

The implications of the finding, from this nr program seem to 
indicate that a mat* lot, is indeed a highly complex learning environment, 
accountability must be defined in a very specific terms relative to each 
student, the teacher, and to the particular device and/or laboratory 
lessons being used. The role of the teacher in mathematics laboratories 
is considerably different from that of a traditional lecturer. The teacher 
training program at San Fernando Valley State College is being altered to 

incorporate the findings from this reasearch. 

He are now beginning our third year in this project, 
deludes two junior high schools and one senior high school, a total of 



about 5000 students*. 
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SOME COMMENTS ON ACCOUNTABILITY 
IN MATHEMATICS EDUCATION 



Edward Griffith Begle 



(Edward Griffith Begle is Professor of Mathe- 
matics Education at Stanford University and 
Director of The School Mathematics Study Group. 
He received both his B.A. aidM.A. degrees from 
University of Michigan, and his Doctorate degr 
■Prnm Princeton University. ) 



, .. v.+o-k- 5 -m+v" when used in discussions of educa- 
te word accountability , er x Qr inst itution (e.g., 

tion, often means that some person ( .g., ^ prescribed instruction) 

school) or educational e ; 8 ;’ SfS lou^t of student progress to- 

la assigned the responsibility for a spec asslgning the blame entirely to 
ward specified goals. This is in conuxa 

the student in ^ iTe s can be specified and tested for much more 

sophisticated^ now than ten ^“[S a go. aTO id careful examination of 

However, in 5 ra0 J aa ? " tests and hence the objectives implicit 
objectives. Instead, we use existing tests and ^ Qf date 

in^ them. This is bad f^ e ™urSl» In Ihe last decade. Also, 
in view of the major changes in math curricui rather than teachers, 

they were designed for standardised tests do not cover the 

schools, or programs. In particular, to provide. 

range of objectives any careful number of schemes for examining ob- 

Jectives have 

objectives ^sTcfgnitive level. Thus, for 

example, SMSG uses this matrix: 



Arith- Algebra Geometry Function 



metic 



Knowledge 
Computation 
Understanding 
Application 
Analysis 



To state the objectives of a particular course, or sequence of 
of courses, the relevant boxes are specified. 
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Of course, in practice each column is split into finer subdivi- 
sions.. For example, arithmetic could be split into whole numbers and frac- 
tions, and. of course there are obvious ways of further subdividing. The finer 
the subdivisions, the more specific the objectives. 

Once the objectives of a particular course of study have been 
decided on, then it is necessary to construct instruments to measure progress 
toward these objectives. This is done by constructing a subtest for each of 
the boxes relevant to our objectives and putting these subtests together to 
form our evaluation test. Naturally, each subtest needs to satisfy the usual 
criteria: internal reliability, discriminating power, etc. This generally 

requires a dozen or more items, which makes it clear that current standardized 

tests are insufficient. They do not contain enough items to take care of all 
boxes. 

You will now complain that such a test as described above is 
too lengthy, that it would take more time for a student to work through it 
than, is available. But now we must remember that we are not interested in as- 
sessing the. success of individual students, but rather of teachers, schools, 
or instructional programs. For these, it is not necessary to administer the 
entire test to each student. Instead, each subtest can be administered to a 
randomly selected subset of the students. The success of the teacher, school, 
or program can now be estimated from the results obtained with the random sub- 
sets. This procedure, called "item sampling," has been known for some time, 
but does not seem to have been widely used. 

Evidence that these procedures are not fanciful dreams comes 
from the fact that they were followed successfully by the National Longitudin- 
al Study of Mathematical Abilities and from the fact that the California State 
Department of Education has commissioned tests of this kind for evaluating the 
State program at grade levels three, six, and eight. 

To summarize, in two important aspects of accountability, set- 
ting clear, sensible goals and using appropriate measuring instruments, we are 
potentially in good shape in mathematics education. We can do a good iob 
along these lines if we want to. 

But let me turn now to another and gloomier picture. Starting 
in the fall of 1962, the School Mathematics Study Group conducted a long term 
study of mathematics achievement. Several tens of thousands of students were 
involved, starting at grade levels four, seven, and ten. These students were 
tested extensively each fall and each spring. In addition, much information 
was gathered about the students' families, schools, communities, and teachers. 

The teacher information was collected in order to follow up the 
vast number of studies previously carried out on teacher effectiveness. It 
included not only the usual background information on education, years of ex- 
perience, amount of recent in-service training, etc., but also attitudes to- 
ward mathematics, toward teaching, toward students, etc. 

0nl y recently have we had time to analyze any of this teacher 
data. . We computed an effectiveness score for each of the teachers involved 
m this study during the 1962-63 school year. The effectiveness score was 
based on. the achievement of the teacher's students during the year, with ini- 
tial achievement and mental ability factored out. 

We were not surprised to find that there was considerable vari- 
ance in these effectiveness scores. This was true for both male and female 
teachers, for each of the three grade levels, and for effectiveness measured 
by student achievement both in computation and in understanding of mathemat- 



I 



ical concepts. research we were not surprised, although we 

in view of past ^earch^we ^ ^ of tea cher at- 

were disappointed since we had h 6 * P information we had collected was 

titudes, to find that in -effectiveness. + _ 

able to account for much of the var background for our next analy- 

All of this is merely zo provj. B had been teaching our 

sis. We found that a sutstantiai^num ®* a ° Mng SQme of our fifth grade stu- 
fourth grade students in 19 & - ^ f 1962-65 seventh grade teachers 

dents in 1963 - 64 . Similar ly a number of the 19 ^ 3 ^ ^ 

in our study were also involved at 8 compare the effectiveness of 

We thus had the opportunity P following yea r. The re- 
teachers in one year with between Year 1 effectiveness and Year 2 

suits surprised us. ihe correlations between^ ^ effective qne yea r we 

effectiveness were uniformly • fj„ a Tiv be effective the next year, 
should not expect that he ^“^“^dings for accountability in mathe- 

The implicationsofthesefinding^ffect^eness of teachers is 
matics education are twofold. First, •vi* ^ ft . e(jaen t ly . what is worse 
not a stable trait, it is nec level we wish, we do not have 

any^suggestion^t^offer^s g^^^g^ri^o^th^teacher^^Poor'per- 

forni^^e S5 «■ — 
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SYMPOSIUM VI 



accountability in social science 



Chairman: 



Dr. Daniel Freudenthal 
Berkeley Unified School District 



Participants : 



Gene Geisler , Professor 
San Francisco State College 



John Westfall, Associate Professor 
Department of Geography 
San Francisco State College 

Lynn Reynolds, Sociologist 
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accountability in the social science 



CURRICULUM AS A SOCIAL PROBLEM 



Lynn M. Reynolds, Jr. 



It is difficult to think 

social science curriculum. In ^e first plac una nticipated intervening 

between the two. Many systems, taken into account before one 

variables associated with elap accountability measures. The two 

as SLrrasfs? "Svs ss«sr«= 

C7SJ£ and*mechanistic connection between the two is largely 

A recently published «- 

of ideas which are singularly approprUtejio^d^ I will take the liberty 
relationships between curriculum scheme. While this procedure may 

of cribbing extensively fro ” ““f^here is no doubt that they will be 
raise as many questions as it answers, tnere 

questions amenable to solution. following: 

Blumer's position is beautifully su^ed up in the following 

Sociologists have erred in ifating problems inol bjec M * Qf 

tions. instead social ^f^^^^hethe/social 
collective definition. This P become legitimated, how 

problems will arise, whether y addressed in 

they are shaped in di «“ ss t J“"’ a “^tituted in putting 

plannedSction^ intoSf feet . Sociological theory must respect 
this process* , 

There is little doubt 

come to be a social problem. It s a p: *° ad nauseum . The "problem" 
administrators, iegisiators , the cou . th — time they are students until 

“ “>• “ ••• ■* 

18(3): 298-306, Winter 1971. 



2 Ibid. , p. 298. 
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the overclass and the underclass, the urbanite and the farmer, the whites, 
blacks, and the third world. The "problem" is pandemic. 

The problem with the "problem" is that it has not yet been put 
in a form which is amenable to solution and at the same time congenial to hu- 
man beings or the institutions they have created. The formulation of prob- 
lems is structured on a model of the particular world under investigation. 

This short essay will use Blumer's model to examine the prob- 
lem posed by curriculum content and development. At the same time it will 
demonstrate, mostly by example, how accountability, pursuing a parallel 
course, follows its own career. At various stages in their development each 
affects the other. These interactions will be typified wherever possible. 

Consider the first stage in the career of a social problem — 
its emergence. To begin with social problems are not an intrinsic malfunc- 
tioning of society, but rather the result of a process by which given condi- 
tions are identified; they simply don't exist for the society until defined 
as such. Social science curriculum has been problematical to those responsi- 
ble for planning it. It is patently impossible to include all human affairs 
in the curriculum. Decisions about what should and should not be included 
have plagued planners for many years. Until recently these decisions had 
been l e f t to the professionals" in the field. The "experts" were their own 
watchdogs. They were responsible only to themselves and to a limited number 
of other constituents, e.g., governing boards issuing administrative fiats, 
legislators, professional associations. Historically, there has been consi- 
derable consensus among these groups. 



Times do, however, change. Nascent and burgeoning social 
movements, notably among blacks, and subsequent reaction have subjected 
nearly every fibre in the fabric of society to scrutiny and often attack. 
These acts have led to postures of rectitude and defensiveness. Social sci- 
ence curriculum has been no exception. New constituencies have arisen and 
have demanded accountability" and they now include not only the experts in 
curriculum and the traditional watchdogs, but also students, ancillary fa- 
culty, counsellors, parents, "taxpayers," special interest groups such as 
organized minorities, and a host of others. In one context or another at 
some time or other each of these constituents demands accountability from the 
others. The criteria for accountability run the gamut from money and time 
through type and qualifications of professional staff and changes in pedago- 
gical procedures. Because of the demands for curriculum revision, because of 
the attempts to meet them, because of the demands for accountability, and be- 
cause of the justifications" submitted in response to the demands, social 

science curriculum — together with other curricula — had emerged as a so- 
cial problem. 



-u ^ „ Th ! SOC * al science curriculum became legitimated as a problem 
with the advent of study groups, master plan committees, and similar aggre- 

8 ** e8 °5 ^ ^ interested in Public education. The important dimension 

added at this stage was the involvement in the planning process of "respec- 

thG conmmnit y- The inclusion of these kinds of people had 
the effect of adding more constituencies and more accountability criteria 
but more important was its legitimating function. ’ 

In recent years this legitimation by study and action-oriented 
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groups led inexorably to discussion of 

s-sstssr - 

about curriculum received . c " rr ®^ slat ^ c sessions. In this dialogue changes 
in committee hearings, and n g d> The important outcome is that vari- 

in curriculum were proposed a PP * iew of soc ial science curriculum 
ous camps, each supporting a P accountability, became mobilized 

s.'siris. «.» •« 

Each of the diverse groups PresentedUs^case^its notions^ 
of accountability to the legislative | 7 lc cou id. From public 

which at this point acted the ^'^^ 0^0 official plans of action 
w«e i wrked°out^ t *lhe*e t ^^ri^ly represent^ compromise - ^result^f 

spect^f or^wha^might * be°workable * concerning curriculum and 

In most parts of C f ““^^^J^i^^mpSented! Some 

how well or poorly the °^ i ^ d P ^ n ® th e rs in this connection. The outcomes 
school districts seem well ahead of others 

remain to be seen. , n The 

The overriding points this paper differential 

social science curriculum is not that constitute the problem, 

definitions of what ought to social problem; but who demands what kind 

2) Accountability per sej s n °^ s° c ial P ca ; riculum , is a problem, 
of accountability from which bo ^ ie ® ’ bility cann ot be thought of as sta- 
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A 



the computer, accountability, and the social scientist 



John E. Westfall 



(John E. Westfall is an Associate Professor of 
Geography at San Francisco State College, an 
has been Director of the SFS ^gi°nal Educat- 
ional Computing Network and Chairman of the SFSC 
Computer Policy Committee. He received his B.A.- 
degree from the University of California, 
Srkeley, and his M.A. and Ph.D. degrees from 
mV. UoeViineton University . ) 



This writer ^rd leaved several 6 quest ions 

"accountability." One reaa °" whom for w hat? As this brief consists of ^ 

unanswered: Who is accoun — __ &nS the college instructor, 

this writer's experiences and view , — will bo use d here is specific to 
As for what and to whom, the statement instructor is accountable to the 
the socTIT sciences: ''The of how hi£ 22^' 

change or be change_d in the futu re. 

This is, of course, simply a SW* 

and may well be criticised for ^i^^tabiSity ^rSinly extends beyond 
instructor's su P e *‘°” ^ h ' says what "degree of understanding 

r p :etsTfnSs in roU a C s d S e 8 scribed bel’ow stem f-m the classroom experiences 
of this writer and his computer-using co g 

First, the computer 

evaluation of student performance tan the teacher of the ironic 

endent of the instructor thus somewhat relief g through examinations, 

task of evaluating the effec tiveness^of hls^own^s^s ter in itself; a 

essays, and so on. The comp . incorrectly, or not work at all. 

ZUTZll interactive germinal 

sis. -its- — * 

81 



heavily on student computer usage, most student evaluation beyond evaluation 
of computer runs, and the presentation, analysis, and interpretation of their 
results, has been found to be redundant by this writer. 

Second, computer -using classes tend to be enthusiastic ones. 
This may seem surprising in view of supposed student antipathy to technology, 
but there are several reasons why students can like the computer. One, of 
course, is the rapid response time of an interactive computer system, where 
the user actually converses with the system. Also, the student is usually 
pleasantly surprised at the ease with which he can learn the rudiments of a 
programming language and begin to make the computer work for him. The 
results of this enthusiasm are improved class morale and such surprising and 
things as finding students voluntarily spending more out-of-class 
time investigating this new tool than once would have been believed possible. 

Such enthusiasm is very desirable, but, it may be argued, not 
central to the aforementioned goal of educating an understanding of social 
processes. Here, the high speed and large "memory" (information storage) of 
modern computers allow the student to investigate in detail, and even try 
variants of his own with, many problems and models r f social systems which 
would be too complex and time-consuming to do in a pre-computer curriculum. 

In this writer's "Population Geography" course, computer simulations of 
social systems through time have been found useful and instructive (eg., one 
study involves changes in land use, population density, and urban:rural 
ratio, in a region undergoing "sequent occupance" by different societies). 

Ot er exercises in th s course involve projections of population numbers, 
characteristics, and distribution into the future. Classes feel that this 
application of the computer is valuable for two reasons--first, current 
concern for our perceived environmental and population problems and, second, 
the realization that the future being projected could possibly be the 
student's future world. In these exercises, a few seconds of "computer time" 
save many hours of student time; for once, the student has time to interpret 
and perhaps even to understand. 

Such is the present status of computing, at least as observed 
at one institution. Computing enthusiasts like to say "this is only the 
beginning," and this is probably true. Nevertheless, computers are expensive 
investments and the future extension of their use in the classroom should be 
considered from a number of aspects, including that of accountability. Even 
taking the narrow (but practical) viewpoint of equating accountability with 
saving the state money," the increased use of the computer has much 
potential; computers can take over at least some of the routine duties of the 
instructor, allowing him more efficiently to teach, rather than to monitor or 
to evaluate. More intellectually justifiable is the possibility of using the 
speed and flexibility of the individual computing terminal to enable the 
instructor to help the student to proceed in his own direction and at his own 
pace to investigate and to develop the "degree of understanding" of society 
called for in the original statement of accountability. 
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SYMPOSIUM VII 



ACCOUNTABILITY IN LANGUAGE ARTS 



Chairman: 



Dr. Harold Weatherbe 
Assistant Research Executive 
California Teachers Association 



Participant: 



Sidney Lester, Director 
Marin Social Sciences Project 
Marin County Board of Education 



Ruminations and Research Related to Accountability 



G. Sidney Lester 



o a dearee from Sen Frencisco St3te Colleg « f q f a 
^gree from aC.L.A. He is a former director of an E.S.E.A. 

Titln III nrnifint^ 



The notion that accountability 

pvtremeiv vulnerable for several reasons. Jh'S vulnerability Tece nt attempts of educators 

bility perse, since th*dysfunctional nature of educational practices, 

ssaffissacssi . . .. „ 

The implementation of acco^tal^ity in educ^tKHi wdl 1^“^ rs hav9 lived with or- 

«* *» simp,v fail t0 observe ,he obv,ous ' 

Upon thesekey f!^™° 
functional* (See figure 1) 



Figure 1. 

Owners-Stockholders 

Managers 

Workers 




BUSINESS MODEL 



Policymakers 

Management 

Production Level 




Boards-Regents-T rustees 
Administrators 
Teachers and Students 



EDUCATION MODEL 



This structure is designed to speed profits to the few individual at the top who have invested 
risk capital in the business venture. 

This mode, is inappropriate for educate '^ f £g£, '" mK- 
educational achievement) ^ model is to serve as support 

tionally and erroneously calted the hierarcny . „ f what students and teachers do, 

Si^shS be'”mspomiv^o" ^he°ne«ssarv C acrhhties carried on bv teachers and students. 
Increased profits in the first ^^^J'g^^profltt^re^e^ltof greater 

AUTONOMY' and^^GE^EXPEi^I^I^'^^slaf ^nds^a^he^prc^^m^l^e^^e^bus^ness 

’ here is support ,r ° m ^ The d,Ch0t ° mV 

is obvious. 
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A more reasonable 
means-ends chain. 



model for education would result from the following accountability 



The Community 
The School Board 
The Superintendent 
The Principal 
The Teacher 
The Student 



is accountable for the SUCCESS of 
is accountable for the SUCCESS of 
is accountable for the SUCCESS of 
is accountable for the SUCCESS of 
is accountable for the SUCCESS of 
who is the measure of the success of the above. 



nr . Accountability would also demand an evaluation system whereby success that i c 



Figure 2. 



Government 

Business 

Education 



PLANNING IMPLEMENTATION 

Legislature Administration-Bureaus 

Own e rs-Stock holders Management-Workers 

Boards-Regents-Trustees Administrators-Teachers 



EVALUATION 

Judical-Voters 

Consumers 



evaluation "ofS^oTihools?^^ ident . i,ial ? le ^ of evaluators. Typically 

s,te,;sss^r ition ot 



demands several modifications of present practice First • goals ofTd * r th6 part ^ f students 
These goals, established by whatever Agency is des’ianat?d ?? ' °l d K? at , IOn ™H st ** established. 
1) must be based on the NEEDS OF STUDENT^n,/!. 6 *! t S re y ons, ble for their Production 
(and outmoded) disciplines or values intendpri mprou/Y* based on the maintenance of traditional 



(See Figure 6 !?"' 11 ''' le9i,ima,e eva,ua,ion of “hoofs must be based on outputs rather than 
FIGURE 3 . ACCOUNTABILITY 



REMOVES THE EMPHASIS FROM INPUTS 

...cost per child 

...pupil teacher ratio 

...new building design 

...recruitment efforts 

...years of staff experience 

...numbers of offerings and activities 

...square footage per child 



AND PLACES THE EMPHASIS ON OUTPUTS 

increased percentage of college entrants 
lower percentage of dropouts 
improved scores on standardized tests 
reduction of vandalism costs 
reduction of anti-social behavior 
increased successful job applicants 
more positive scores on school attitude surveys 
peater community support for schools 



inputs. 
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Finally, successful and accountable 

-S-SKT-SS^ 

power to fight only for selfish gains. 

Accountability will work in education “^^^“^i^as^ofiS^s responsible 
foMurtify^'T^E^R^uatams and^la^prov^ded the time and support necessary to make 
professional decisions. 
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SYMPOSIUM VIII 



accountability in vocational education 



Chairman: 


Dr. Jerry C. Gar lock 
Coordinator of Research 
El Camino Community College 
Torrance 


Participants : 


Wayne L. Butterbaugh, Superintendent- 
Southern California Regional Occupation Center 

Torrance 

Barry Jensen, Vice President 
General Behavior Systems 
Torrance 

Richard S. Nelson 
Chief, Program Operation Unit 
Vocational Education Section 
State Department of Education 



accountability and the individual learner 
in VOCATIONAL education 
THE S.C.R.O. C. MODEL 

Wayne L. Butterbaugh, Ed. D 



Accountability in the broadest sense may be re- 
lated to innumerable variables in education, such as instructor 
accountability, student accountability, institutional accountability, 

and fiscal accountability. 

The Southern California Regional Occupational Center 
also been cognizant of another element in the accountability 

Sr 

where no one need fail* 

The Southern California Regional Occupational Center 
located in Torrance, California. The Center provides occupa- 

tf their regular high school program. As an adjunct we also tram 
adult students when our facilities are not being used by tte hg h ^ 
school population. This fundamental objective is keyed y 

the concept of accountability in education. 

To meet the needs of the individual learner, the 

_ «.• ratal r^ntpr ha s developed a systematic approach to tn p 

Occupational Center has deveiope y j ent and course 

cess encompassing course selection, cour P . b e- 

implementation, as well as counseling and placement procedures 

fore and after instruction. 
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The following is an outline of the steps which are 
taken when a new course is selected, developed, and implemented: 

1 • The need for the course is established. 

This utilizes a three -fold approach. First, a survey of national 
occupational trends. The question to be answered here is: Will this 

course provide potential students with a skill which will be salable 
on a nation-wide basis in the future? Second , a survey of local 
industry job needs. Does local industry in the area to which the 
Center directly is responsible have the need for personnel trained in 
the identified area? Third , a survey of student interest in the course 
to be offered. Is this a course which the student will find attractive; 
will we have enough students to financially justify its operation? The 
primary purpose of this rather rigorous selection process is to meet 
the needs of the individual learner, in terms of future job prospects 
nation-wide, in terms of local job availability, and in terms of ex- 
pressed student interest in the job for which instruction is to be 
provided. Thus, accountability to the individual learner begins with 
the selection of a particular course to be offered. 

2 * A course Advisory Committee is established. 

The course Advisory Committee is composed of representatives from 
local industry who will ultimately be hiring graduates of the instruc- 
tional program. This committee determines what general areas of 
the instructional program must be encompassed in order to provide 
personnel who will have the specific skills required for employment. 

Here, accountability to the individual is in the form of a guarantee 
that the instructional program in which they participate will provide 
them with a skill or skills that are viewed as valuable by their 
potential employers. 

3 * .Program requir ements are identified and a Cost Effec tiveness 
Analysis is performed. 

Based on the recommendations of the course Advisory Committee, 
necessary equipment, personnel, and space requirements are identi- 
fy . A cost effectiveness study of these program requirements is 
performed to ensure that each piece of equipment will provide the 
most efficient means of quality instruction. 

Accountability to the individual learner here is in the form of quality 
of instruction. When technological advances mandate a newer piece 



. and a possible additional initial capital outlay, such 

equipment 6 wiil be acquired if it can be established that mastery of 
this equipment will be necessary for later job placement. 

It is “d»ring colrTe" design and development that^ systems 

Dictionary X ^onaljitles 
designation or designations is then selected on the basis of the 
general course outline which has been provided by the Advisory 

Committee. 

In the Center's current programs, there are some 600 to 700 DOT'S 

^at B :^DO?Teir a sp a e V c«i 7 c° 

Iry-le'veYTol and since many of the DOT'S are in ascending order 
of difficulty 3 provides one means whereby we can guarantee tha 
Iverfsmdem irrespective of individual differences is successful 
Some trainees may be able to meet only one or d OT 

level DOT'S in a class. Others may be able 

requirements at the end of their program. However, “» ster J 

„ * DOX area 0 f a course represents acquisition of a salable 

£r crr= 

fic areas to be covered in the instructional program. 

5 . T.rminal Performance O bjectives and other course support 
documents are produced ^ Terminal Performance Objec- 

5“*“ : h eVerd S V U th: c;u:re C fd C viL e r r y m Committee to ensure that 
all -ecuisite skTu areas are covered and also that no extraneous or 

— rrrvaoped^or acqmred for 
the course il produced. Additional instructional support materia 
are identified, designed, and produced. 



At this point the course selection and development phase is complete. 
Each individual student is assured of an efficient instructional program 
that will provide him with specific skills leading to a specific salable 

job, and one for which jobs, locally and nation-wide, are currently 
available. y 



6. Identification/selection of course enrole es. 

Based on the DOT job descriptions, prerequisite skills and abilities 
are identified which are necessary for successful job performance. 
Only those attributes which are mandatory for successful job per- 
formance are specified. For example, the ability to hear is not a 
requirement for a keypunch operator. Therefore, students who are 
hard of hearing or deaf may reasonably be admitted to the Keypunch 
course and still be assured, if they master the skills of the job, 
they can perform the job. Requisite attributes are compiled to form 
a course prerequisite description. Counselors, both at the Occupa- 
tional Center and at the Center's member high schools, use these 
prerequisites and the students past performance in related courses as 
well as various employer, counselor, and teacher recommendations 
to assist the student in the selection of a specific course that will 
meet his needs and abilities. This process has resulted in a 90% 
rate of successful trainee course completion at the Center. 



In terms of accountability to the individual student at the Center, 
this rather thorough process of counseling ensures that each student 
will select a course which meets his specific needs and abilities 
No student is encouraged to enroll in a course in which he cannot 
e successful m terms of later job performance, and no student is 
encouraged to enroll in a course where all data indicate that he will 
later become dissatisfied and unhappy. 



Course implementation and instructio n. 

At the time of initial course implementation several items are pro- 
vided. The course terminal performance objectives have been writ- 
ten. A competent instructor, one who has worked in the area to be 
taught for a minimum of five years (and most have 10 or more years 
experience) and who has recent experience in the area, has been 
acquired. The bulk of the course support materials has been pro- 
duced. What remains now is what may be termed "classroom man 
agement. " . Each course at the Center has an enrollment of approxi- 
® X individuals. Each individual in each course is different, 
ac individual in each course learns and performs at a different 
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rate, and in a different manner. Each individual in each course 
would like to finish the course, receive a Certificate of Proficiency 
and obtain a job after graduation. In course implementation and 
instruction, accountability to each of these individuals is mandatory 
and may be measured in successful course completion and job ac- 
quisition. 

To meet this end, each student is permitted to work at his or her 
own rate. A "hands -on M approach, one which is directed towards 
performance of skills that will be required later on the job is 
utilized. Theory, written examinations and assorted useless paper- 
work is kept to a minimum or eliminated entirely. Course Perfor- 
mance Objectives and the continuum of entry level positions is de- 
lineated early in the course. All activity in the course on the part 
of the students and the instructor is directed toward meeting per- 
formance objectives and, thus, job requirements. Wherever possible 
a variety of mediated instructional tools are produced or acquired. 
These media modules are made as self -instructional as possible, so 
that the gifted students may speed ahead, while the slower students 
may concentrate on one area until they have mastered it. In course 
implemention self-instruction, individualized instruction, and perfor- 
mance mastery present the case, while the judge and jury of success 
are individual job placement and performance. 

8. Course Revision and updating. 

During this phase of the course development process the entire proce- 
dure is repeated. As required during the school year and at the end 
of each year, each course at the Center is examined both in terms of 
individual student interest and performance and the current job mar- 
ket. Courses leading to skills which are becoming obsolete are de- 
leted or drastically altered. Instructional programs which have not 
produced successful students are scrutinized and revised. Instructors 
who have not implemented an effective instructional program are 
provided with in-service training and, where necessary, additional 
instructional support materials. In short, the concept of account- 
ability to the individual learner at the Center encompasses the entire 
operation of the Center; where every student may succeed and leave 
high school with a marketable skill. With this thrust, secondary 
education will achieve two lofty goals - education that is relevant and 
education that is accountable to the tax paying public who support 
our schools. 
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ACCOUNTABILITY, ASSESSMENT, AND RELEVANCE 
IN VOCATIONAL EDUCATION 
Richard S. Nelson 



(Richard S. Nelson is Chief, Program Operation 
Unit, Vocational Education Section, California 
State Department of Education, Sacramento. He 
received his B.V.E. degree and M.A. degree 
from San Diego State College.) 



Accountability, assessment, and relevance are key words in the 
effective administration of California's program of vocational education. 
Vocational education must be conceptualized as a life-long program of career 
development. It now, more than ever before, must encompass significant so- 
cioeconomic concepts. 

*. .. . , 0r ] e of its major roles is preparing all citizens— especially 

the di sadvan taged— to move from the possible misery of poverty and unemploy- 
ment or underemployment to recognition through social mobility and meaning- 
ful employment and advancement up the career ladder. 

Crucial to continuing and expanding vocational education ca- 
reer development programs for all persons-in all geographic areas— in all 
occupational areas— is the need for program flexibility that will provide 
multiple options for youth and adults in making career choices and changes. 

No longer can thousands upon thousands of California's young 
men and women continue to leave high school with or without a diploma — un- 
fitted for employment— unabl e or unwilling to go on to a community colleqe, 
state college, the university, adult school , private school or college. As 
we ponder thoughtfully the growing charade of "irrelevance" in our schools 
and in our colleges, let's join the U.S. Commissioner of Education in his 
point of view that school administrators are preoccupied with "college en- 
trance examinations" and that "general education" is a necessary requirement 

of success for all young people. Let's purge ourselves of this concept of 
academic snobbery. 



. .. x 1 contend that vocational education is a responsible manpower 

delivery system that has performed well, not only in previous times of emer- 
gency, but through peace-time emphasis in the training and retraining of 

WorldHar 1 -- World War II— the Korean War-and now to our return- 
ing veterans of the Vietnam War. 



Tf)® P^sfint federal -state- local system of vocational educatior 
offers a responsible and accountable delivery system that can appreciably 
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assist our state and nation find solutions to the massive social and economic 
problems that we now face. 

provides for: 

• s,tasr^ss " 

• wnssrJsa - 

occupations 

• Post-high school career preparation for the technical and para- 
professional occupations 

• consume^educa 1 1 on* needs MVoft e'wi tWJS 1 sTpSnlhose' 
living in disadvantaged areas 

• Provisions for adjustment ^^{ r Jeed°such as S ?he unemployed, 

^^v.i3s*r5ssxi - ■"»>< 

« r**- -~'S,X;, lr S2 ESTlSSSS" “ *- 

veloping educational programs, activities, an 

• Instructional cost 

• Equipment and supply cost 

• Administration and teacher education 

• Construction cost 

■ Financial assistance to those who need such assistance to stay 
enrolled in vocational education programs 

• Evaluative and accountability services 

The persons in California 

for vocational educati on must be count* d jjjj^j^depend upon four years 
out of ten of California's population is now ame w [l esult K is that no less 

of 'college preparation for 0 ““ p inh°hol ders are^prospective recipients of the 

services of vilaiiUlISlon^s provided in California's public schools. 
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il .. - , a state where approximately eight million persons are in 

left to chaice° r Not^nl^m ?5 q ^ sl * tio ; of employability skills cannot be 
iw.iitc chance j Not on Jy must thousands upon thousands of persons— youth and 
f/nal tS> 3 en and women "" be systematically prepared for initial employment each 

upqrad^their^kilu^anH - ^ already employed must continually 

pgrade their skills and knowledge in order to maintain employment stability. 

. n ^ J he most n ’9orous and time consuming part of develoDina a 

e r ;i* M***"**- *" d relevance 

education is that of program analysis. The first step is to determine the 
major functions that must be undertaken in developing the rationale There 
are three distinct dimensions of inquiry inherent in this anal^is 

1 ' IwrF h nnirrTrS£c Ca 2 IJ be da "? nst r at ® d that the GOALS and PERFORM- 
ANCE OBJECTIVES of the vocational education system are RELEVANT 

„ t °ttirrL ten V? d com P?lli' n 9 social, cultural, and economic 
problems of the state, region, and nation? 

2 ‘ an 2 P erforma nce objectives to pursue, to what 

?n t ^ + C K n - lt ^demonstrated that these goals and objectives are 
in fact being achieved by the system, both short and long term? 

3. How can the system provide ACCOUNTABILITY while achieving its 
goals? To what extent are resources used optimumlv? To what 
extent are individuals benefiting from the system's programs, 
services, activities, and organization? 

. .. . £ . Accountability for whom? For what? Accountability is the 

condition of being accountable, liable, or responsible. y 

. . Any discussion about accountability which includes Derformanrp 

Knu^ a L tl h n9 i pe r°T nCe 1ncent1ves - "*r1t salaries, and ihe Icher p?an is 
outcomes. h6 ^ P educators th1nk more P recise1y about their goals and the desired 



tional education, but"!? S?S*i? n «SLS!S2f?,S ,B V0Ca ' 

in educational ^ contributed to the «■"*» <"terest 



• Federal emphasis on evaluation of school systems and their programs 

• Review of education in terms of cost effectiveness 

• ”j|ey n serve 001 systems responsive to the clientele and communities 



.. Mh ? t means is that the management of vocational education 
at the state, regional, and local level must change and be accountable while 
producing a better product through the effective ise of availaM? resources 

«?n^ U DESI U RL^c n OMES USTIC G ° ALS ‘ ° BJECTIVK ’ and a " 
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Some definition of terms is needed at this point. 

A REALISTIC GOAL is a staLementofbroaddirection, 

i^nof conc^ventth 0 ! specific achievement within a specified 
time period. 

A g^ 1 ^ eSi i^i^VSV^"e objective SSi 

be developed. 

A DESIRED OUTCOME is a measurable result of planned activities and 
achievement. 

done through Instituting a system of differentiated staffing. 

VrcoTcf sr^^ s^rtionVrfctiVi tles^eading ' f ieS ’ 

urable results. 

Why goal setting? 

Total organization is involved in a common purpose 

Decision-makers at all levels are forced to review purpose, responsi- 
bility, and relative importance of program 

• Performance objectives are established, requiring accountability of 
attainment 

. Means are provided of .assuring goals are translated into performance 
objectives and specific tasks 

m u arinnv nf our world today comes not only from the 

awareness of but vi paucity of solid answers. 

There is a crisis in international relationships 
places* St T he^cri siskin '2Z%3& ~ 

5dSS;. Ind in our race relations-these things disturb us. 

It mav be helpful to know that the Chinese translate the word 
"crisis" by two characters meaning "danger" and "opportunity. 
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danger is *> 
educational ivery tytt™ J^" 9 

Cal iforma will assure accountability, assessment, and relevance 

’• encfof"ali e pMp?i 0n mUSt beC ° me 9 Part ° f the educational experl- 

2. Vocational education must be a principal element in career educa- 
tion in kindergarten through adult education programs 






Introduce the world of career to all school children grades 4-9 

aMitea^rn^J^f 06 ex P loratl ‘ on ! counseling, and guidance as 
education Part f er educat10n 9rades 4 through adult 

* sernnHav'E re i!Sne5 niCa ^ * . vocatl * ona l > and technical proqrams at the 
secondary, post-secondary, and adult levels 

* secondary hand^adul t"l eveT ’ ^ " 9 opportunities at P° st ' 

3 ' state!°and market"" 6 t0 be reSP °" Sive t0 the local > 

4 ' for youth and 

5 ' sys te^of^oca tional ’education? * b6 *" f " te9ral Part ° f the sctool ' s 

6 ‘ !°“?!°a" al . e ^ ca ‘ 10 ? ? outh 9 rou P s must continue to expand and be 
involved m the decision-making process of career education. 

^ ' P ^J[ a * e sc |)ools and private industry must continue to be an intearal 
partner and part of vocational education programs? 9 

8 ’ vocational 6 e ducation? S Wl ” be Ut111Z6d in the further «<>" «f 

prepared for oc™pati?n?Mife ^keeping with 6 the inabilities; aptittlde^ and 
skillTand understandings? Chan9ln9 “orld-adults with improved occupational 

, L1 . , Assessment Is needed of the product* and of the edura*Hnnai 

an^iildnp" 1 ^- 3 -^ *? °P® ratl * on * assessment I mean to analyze critically 
definitively the nature, significance, and statu^of merit of the 
educational endeavor. The responsibility for this appraisal task Is shaded 
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by the administrators, the supervisors, directors and coordinators, and the 
teachers. 



Vocational education institutions, whether they be high 
schools, community colleges, or adult schools, must constantly evaluate i 
theyare to™ in business. If they reach the point where their product 
no lonaer achieves the skills and knowledges necessary to successfully enter 
TntoTnd compete In the labor market, they are replaced by more desirable 

institutions. 



INDUSTRY’S 



EXPECTATION OF EDUCATION 



Barry T. Jensen 



fRarrv T Jensen is Vice President and Director of Behavioral 
Science at General Behavioral Systems, Inc. He received h.s 
B. S. Degree in Education from Brigham V “ n | “”' rS ‘ * 
and his M. A. in Education and his Ph. D. in Psychology 

Ohio State University. ) 

Describing what industry expects of *vo-' 

ment of the functions to It 

isal^o necessary to define the functioning; for instance vocation- 
QttiT^arHs or desired levels of attainment should be set. A 
accuracy and other characteristics. 

asx z-.tr, 

job skills training? 

In an effort to get some information, I solicited 100 California 

that vocational training is a function of the school and nea y 
half said that citizenship development is a function. 

Thev were also asked to define vocational training in terms of 

what it should include and to indicate at J^^hes expenses, 
it should begin. There was great variation in these re p 
Development of work attitudes were seen as expectation . 



I mention the questionnaire in order to illustrate the range and 
variety of objectives which industry holds for education. Most 
of industry's expectations deal with the affective domain of the 
taxonomy of educational objectives. 

Some of what industry wants schools to do is to compensate 
for industry's failures, in the same vein that teachers often 
blame the home for the school's failures. In my little survey, 
development of punctuality was the most commonly mentioned 
expectation. I don't know of another objective in the affective 
domain which the schools emphasize so much. I'm not pro- 
nouncing judgment as to the success of the school in this area. 
But it does point up one possible weakness of the businessman -- 
that of setting and enforcing a standard. The school can lay a 
foundation of behavior but unless industry enforces its own 
standards in this respect, what should we really expect. 

Schools are educating the workers of tomorrow. The in- 
creased emphasis of the U. S. Office of Education and the 
State Department indicate, to me, Educator's acceptance of 
this fact. This offer of Education and L. A. City have the 
objective that every high school graduates have a marketable 
skill. What jobs should we train for? I imagine that if we 
asked a group of industrialists they couldn't tell what specific 
jobs to train for. We can be sure of one thing, however; 
there'll always be a need for communicators, e. g. , a secre- 
tary. She might use a voice -writer or other device to replace 
shorthand just as today she uses a xerox machine instead of 
carbon papers. So perhaps we should train secretarial people 
m the basic skills -- the art of filing and competence with 
machines. 

The machinist's job is changing; we can look forward to radi- 
cal adjustments having at least as great effects of production 
as the development of numerical control and automatic ma- 
chine tools. So, the machine shop man-to-be should, become 
a reasonably competent with a range of equipment and tools 
so that he has a base from which to learn modifications -- he 
should be taught how to learn. 

I am mindful of the fact that not all people are equally intel- 
ligent but I am also convinced that, given a properly stimula- 
ting environment, a youngster can do much more than we now 

expect and that even average youth can learn and to solve prob- 
lems. 
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SYMPOSIUM IX 



ACCOUNTABILITY IN PUPIL PERSONNEL SERVICES 



Chairman: 



Dr. Lester W. Ristow, Assistant Director 
Division of Research and Pupil 
Personnel Services 
Los Angeles County Schools 



Participants : 



Thomas W. Smith , „ . 

Director of Research and Pupil Personnel Services 

Covina Valley Unified School District 



Stuart J. Mandell .... . 

Director of Guidance and Supervisor of Child 

Welfare and Attendance 

East Whittier City School District 



,? 
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guidelines for accountability in 

PUPIL PERSONNEL SERVICES IN CALIFORNIA 
Thomas W . Smith 



(Thomas W. Smith is Director of Research and 
pupil Personnel Services for the Covina 
Valley Unified School District, Covina, 
California. He received his B.A. and M.A. 
degrees from Stanford University and his 
Ed.D. from University of Southern 

if 4a s 



There has been a serious loss of public confidence in 

American education during ^.^/“Sie.VtacJeas^nu^er of 
problems have increased locaUyand nationwide, an^ ^ ^ , good 

citizens have expressed do p “ bUc dou bt has been an increase in 

job. A concomitant of this rising P , t u at various educational 

the pressures put on schools to p«duce desirable 

programs, including programs in pup state Department of 

outcomes in student workers to prepare a Process 

Education organized a team of p P P personnel services. 

p.nide for the Development of G * ide was revised and 

S2SJ 2-bJSSSS-^ California Personal and Guidance 

Association. 1 

The purpose of this monograph is “ ^aselTpupil 8 personnel 
procedures involved in the devel “^tSof mgest. ttot the 
d^TSThiec&rer^slacilitate both effective planning 





107 

90 



* 



^ £or 

school guidance programs: 

1 Ob iectives- based programs focus more directly on 

who need further or modified experiences 
desired outcomes. 

3. Objectives-based enable school 

visible the successes of their programs uiw. & 
of explicit outcomes. 

4 Objectives-based guidance programs offer systematic 

methods for evaluating on-going programs and planm g 
improvements that will retain activities that are 
effective and will discontinue or modify ineffective 

procedures • 

There are four major steps involved “ operatio^of^n 
effective objectives-based pupil personnel program. These p 
follows : 

s,-s- ss- 

personnel services staff. 

, c e i e ct and present to students experiences and 

information designed to assist them in attaining 
their desired outcomes. 

3 Assess the performance of students to determine the 

* n f the experiences and to identify students 

:ho e do nft o«a“ P “e or more of their desired outcomes. 

4 Provide additional experiences to promote the attain- 
ment of outcomes for those students *» d ““ b ln 
their anticipated outcomes from original experience. 

The Process Guide describes procedures for assessing student 

needs and developing pupil i^exalples^f local district 

objectives and assessing outcomes. Spec it ic examp 

procedures and outcomes are provided. 
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$TEP jo PUPIL PERSONNEL SERVICES 



OPERATIONAL OBJECTIVES: FIRST 

ACCOUNTABI LITY 



Stuart J. Mandel 1 



(Stuart J Mandel 1 is Director of Research and 
Guidance^of the East Whittier City choo D.s; 
trict Whittier, California. He also serves as 
Adjunct Associate Professor at Un^S'ty of 
Southern California. He received both his A.B. 
and H.A. degrees in psychology ^om Occident 
College, and his Ph.D. in Educational Psychol 
ogy from University of Southern Cal, forma.) 

r.ntra 1 to the idea of "Accountability is the notion of "show 
me. r. from Missouri ." Thl s^lea for -creteness verifies-^ 

tion of results has recently em 9 „ ii outC ome-based management," 

based approach," 1 management: Y ^ accountability oriented descriptors 

"resul ts-management ," etc. All ot tn hether soma verifiable (observ- 

imply: 0) decision-making on *l*™**p* (2) a concern for a fixed end 

able or measurable) end is being pp /Lv g conce rn for strengthening 

with variable means rather j^ n r aiher than’merely mai ntaini ng it; (4) a 
the output of the organ izat ion n , s ^describe, the more error we 

ZZllrU* recognition Et?r£2£.-n with process can impede the 

development of the product. 

This concern for accountabi 1 ity achieved through ^objectives 

defined in verifiable terms J»s ^f,,!^f e * e s ^ r f !^derstanding" or "greater 

?r^ti v = 

for today's models which reqmremaxiwm 0^601^1^ impossible t0 

served progress toward objectives. S m , , object jves, writing 

Z :r^:rbS“!^: n ?n°^crei: 9 ;e;ms does reduce the chance that we 
will fool ourselves or others. 

If we are to obtain 7j- m s P-^r r ; s th a T e an S S ^retranslating 
building on the gains of past. To J the present> Th is is usually 

the discourse of the past in 0 abstract to the most concrete. A model for 

an exercise in going from the most a dance w j t h their abstractness 

ordering and classifying '^ es us to utir ,ze long established value 

gives us two advantages. (1) ,t: l| en ®,'„! (2 \ it oermits the development of 

Srss - «- — - - 

suring "accountability." 
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goa 1 s 
forms . 
leve 1 s 
viewed 
occurs 



The model below shows steps which may be taken to translate 
or objectives from abstract non-behavi ora 1 to concrete behavioral 
It shows how the levels of abstraction, levels of objectives, and 
of decision-making are interrelated. These relations should be 
as relative rather than absolute. In actual practice, much overlap 
in the process of decision-making. 



FOUR LEVELS OF RELATIONS AMONG OBJECTIVES 



(0 

( 2 ) 

( 3 ) 

( 4 ) 



Level of 
Abstraction 




Level of 
Obiect i ves 




Level of 
Manaqemen t 


i 

Abst ract 


t 

Pol i cy 


. 1 

Legislature and 


| 


f 


Board 


Semi -Abstract 


Program 


Administ rat i on 


I 


I 


Supervi s ion 

* 


S' 

Semi -Concrete 

t 


N/ 

Operational 

f 


l 

PPS Staff 
A 


Concrete 


J/ 

Behaviora 1 


I 

Student 



... . In addlt| on to providing a way of going from broad policy to 

specific behavior, the model is also useful for illustrating the division 
of authority and accountability in a school district. Hence, a logical way 
of allocating deci s ion-making can be defined so as to achieve "management by 
objectives." n this way, a strategy can be outl ined to help the student 
attain the goals set by society for the institution. A basis for priority- 
setting can be provided at each level by reference to the one above it. 

Hence level one, the ^oMcy level, defj nes the types of decisions appropriate 
for local and state boards, the State Department of Education, the Legisla- 
ture, and community. At the program level , the administration of a system 
would have pr ime respons i b i U ty for decision-making. At the operational 
level, it would be the pupil personnel services staff; and, at the behavioral 
Je^i, primary responsibility would be retained by the pupil. Hopef ully too 
ment^tT COntinually adjusted in accordance with current needs assess- 



or .. . ,! n considering the development of objectives for the purpose 

bl k^ S '° n "^ km9 D 'L' ,S im P° rtant that the Parallelism of these processes 
be kept in mind. Problems arise when PPS staffers try to plan and measure 

their work using broad measures and program level statements of objectives 
In such a case, the authority to modify activities, schedules, equipment, 

Innnoc ’ e tc ‘ ; necessa r y to produce change will be lacking. As an example, 
suppose a counselor were to propose "to strengthen self-confidence and in- 
crease academic motivation by providing eight hours of group counseling 
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(eight maximum per group) to all \ to^much for one coun- 

b! r f 1 :^? U ^ y w b o:r d U n:t 3 L^ 0 ^l --^staff to meet the deman, 

A better strategy would be for t !?® 

two hours a day for eight weeks of r 9 e°ght groups by’october 1 using the 
invited, screened, and selected f 9 9 Jn g pos ] t i on to develop 

following criteria The couns which can be defined at the 

and/or select objectives for _ th ® J Saviors or activities he will be respon- 
operat i ona 1 level in terms of wh behavioral level objectives can then 

sible for in thTjroup. F “ rthe r^ h/^rt icIpantU-M .e Ji th the coun- 
be developed in collaboration wi P s | b 1 1 1 ty away from student) 

selor facmtatjna not «SW1&' ^"YrSps Sin UZ be defined so as to 

behavior. Performance criteria desi rable behavior change 

be semi -concrete statements of inst stated in formative terms 

e. g., 80% wi 1 1 show 9 alns * ^tharTarrWal as shown by the instrument chosen 
WhiC e J ef aS? wili r shL r gains in the use of "I can- over -M can't- when asked 
to* ra t e * a^J f s t of activities which they feel are important. 
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SYMPOSIUM X 



performance CONTRACTING 



Chairman : 



Dr. William Watts 
Associate Professor 
Department of Education 
University of California, 



Berkeley 



Participants : 



Ernest A. Poore 

County Superintendent of Schools 
Fresno 



William Booth, Project Director 
Office of Economic Opportunity 
Fresno City Unified 
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A ONE YEAR EXPERIENCE IN PERFORMANCE CONTRACTING 



William P. Booth 

(William Booth is presently Assistant Principal 
of Fresno High School, Fresno, 
served last year as the Director of ^he Of i 
of Economic Opportunity Experiment in Perf or 
mance Contracting in the Fresno City u nif 
School District. He received his B.A. degree 
from U.C.L.A., his M.A. Degree from Fresno 
State College, and his Doctorate from the 
University of Southern California.; 

Performance oontracting in educa^ is a^egal relationship 
in which one entity contracts with an t ( educational function for 

swss 'cssasrsJ; 

.‘S— ... .... *•“"* v 

tests. 

+11fl l lv everv educator has an opinion on the subject, and 

most are held with vehemence, only 

restructuring of our public e uc - d L anv substantial experience in 

the Dorsett-Texarkana experiment Provided ject are still being 

performance contracting, and oom 1 ^ 0 ^ re snooty* Unified School District 
debated. The experiment in which the Fresno WJH** ^ attemptj fun ded 

participated during the past schoo y , , body of substantial 

by the Office of Economic ; ^IStoiJtt «iSle assisSd in making rational 

decisions^ this Sea in «hich public pressure might well become acute. 

Each of twenty-one school ^^/“^|^^°one 

Florida, was provided approximately 3 > . teach reading and math 

of six O.E.O. selected performance contractors Ld 9. Students 

skills to 600 students, 100 each in gra ^^es , , > s» > sub j ects> an d from 
selected were below average in Payment to the subcontrac- 

schools in which minority students P r ^^^ f f a ^nces on instruments 
tor in each case was based upon P° ^ 8 f 0 . s 2ec?ed contractor. The 

selected by O.E.O. and administered by an 0 C series of contro l 

over-all program evaluation, * n * basig as those within the experi- 

schools with st ^ den ^ the basiHontract involving the O.E.O., the 

mental group. In ■ addit: ion Fregno , s casej t he Westinghouse Learning 

Fresno School District, and private firms for purposes of 

Corporation, separate contracts^ were e ; l ulation 0 f final payments, 

student selection, management support, and caicuxa^i 
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The Westinghouse system employed a behavioral objective 
oriented series of skill sequences, programmed in structure. Materials 
were selected from all available sources j few were created by Westinghouse. 
The only hardware employed were cassette tape recorders and nominal use of 
Bell and Howell Language Masters. The basic operational philosophy of the 
Westinghouse learning system is what the corporation terms "contingency 
management", in which positive activities are rewarded with "points" re- 
deemable for small premium items • These incentives were used primarily 
as rewards for positive classroom behavior rather than for excellence of 
academic performance. 

It is most unfortunate that at this writing the final post- 
test results are not available; my evaluative comments, therefore, must be 
in terms other than those basic to the planned evaluative scheme. In terms 
of our year* s experience, and in the situation in which we operated, several 
positive aspects of performance contracting, or private involvement in 
public education, were in evidence: 

1» Private industry has capital resources far beyond those of any 
single school district, even with governmental assistance, for the develop- 
ment of educational curriculum "systems". 

2. The potential conciseness of performance contracting systems in 
terms of contract definitions, limitations, evaluative systems, and what 
has so far been the "rifle" approach to curriculum direction may be helpful 
in our development toward the thus far elusive concept of accountability. 

3. Private contractors have personnel control to a greater degree 
than do public educators. This, of course, has potential for misuse. 

h* Private entities, at least in our limited experience, are in a 
position to make change more rapidly than are we, within strictures defined 
by contract. 



On the other hand, performance contracting, or private in- 
volvement in public education carries potential for some severe problems, 
among them: 

1. An almost unavoidable conflict in authority and responsibility in 
the daily operation of the school, assuming simultaneous public and private 
involvement. To what degree may a school principal, for instance, inter- 
fere with the operation of a contract class when he feels the best interests 
of the child are not being served. A contractor could argue that the suc- 
cess of his system, and his. eventual payment, could be jeopardized by such 
interference. 

2 . The separation of total school staff into contract-involved and 
non-contract involved groups, particularly if non school staff are employed. 

3. The validity of the instrument of measurement is likely to be in 
question, no matter what it happens to be. The question ceases to be aca- 
demic when payment is involved*. 

On the basis of the year's experience, our recommendations are: 

1. Pursue performance contracting when you are sure that your own re- 
sources cannot fill the need, and that the contractor can. 

2. Pursue performance contracting when you have a defined need, not 
when it appears an easy way out - it may not be. 

3* Spend time on contract terms - in the long run it may be economical. 
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PERFORMANCE CONTRACTING 



Ernest A. Poore 



(Ernest A. Poore is County Superintendent of 
Schools, Fresno, California. He received his 
AB degree and graduate work from Fresno State 
College. He has been a coordinator of tests, 
guidance and research; director of special 
services; assistant superintendent for edu- 
cation; and has served on state committees 
implementing state mandated testing programs. 

He is currently the chairman of the ACS 
standing committee on research and development.) 



Performance Contracting, the larger issue of accountability, 

. n . tonic and key issue of many national, state and professional 

has b66ti the topic j a dstrinifiiit to edu- 

cational'progres s^a t£e-^f^ 

by^tated^behavioral^b jectives th” °are measurable within a predetermined 
period of time and a given achievement level. 

Reeardless of how you view Performance Contracting, it is 
pntial that we as educators be aware and knowledgeable of its po en 
( Wli»e to be educational leaders. The performance 

contractin^movemen^mustTbe viewed - « S^aSSK - 

" rth4 d^d1“n:trab e ie1asults from the teaching-laaming 
process. 

T1lprp can be uttle doubt that federal assistance programs 

.... s: H - rr.SX i.’SKZ-Jte. 
irs£ sas u — - — 

mining the^ro duct or proof of results-input to output. 

In this presentation, I shall not endeavor to weigh the ad- 
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would prefer to provide you with an outline and check list for your con 
sideration, if and when called upon to discuss Performance Contracting. 



Part I 


- Contract 


Part II 


1.00 General Purpose 

1.01 Definitions 

1.02(a) Statement of Work - General 
1.02(b) Statement of Work - Specific 

1.03 Period of Contractual Obligation 

1.04 Relationship of Contractor to Subcontract 

1.05 Evaluation and Follow-up 

- Criterion for Selecting Subcontractors 


Part III 


- Monitoring and Payment Schedule 


Part IV 


- Evaluation and Follow-up 



SYMPOSIUM XI 



ACCOUNTABILITY IN HIGHER EDUCATION 



Chairman: Dr. Wallace R. Muelder 

Associate Dean 
School of Education 
University of Southern California 

Participants: Leslie Wilbur, Professor 

School of Education 
University of Southern California 



Kenneth Weisbrod 

Associate Dean, Counseling 

California State College at Long Beach 

Roger A. Kaufman, Professor 
U. S. International University 
San Diego 
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accountability and the curriculum 



Kenneth C. Weisbrod 



Kenneth C. Weisbrod Is Assoc ate Dean of 
Counseling and Testing at California State 
College, Long Beach. He received his B.A. 
degree from the University of Red ands, his 
M.A. degree from Stanford Uni ve rslty, 
his Ed.D. from the University of Maryland. 

One does not need to be an expert observer of the ^hanging 
scene In higher educetion to be .^ 

entering an ^bge of account abi I ity. q . . { the most efficient means of 

college level ere: whet 1 characteristics of *thestudent population; whet Is 

instruction; what are the chara perceived by the students, 

the relationship between P^°9™ ^ b +he lns titutlon; what does society 

expect” o° 9 t he grSe” endTe? evidence is there thet current curricula* 
modi fleet Ions will enswer these end other questions. 

_ u , (+ in nmnhnsls seems to be from whet Is to be taught, to 

most" 9 f M c I entlyt 9U New C emphes”s + requ"l re^e of + the circle- 

ul um-l earner axis* 

The cel. for eccounteb, I llty se^is to ^natural 

of the bulging wells of c 1 B !%?^that we Increese the speed through the 
to the campus end theexpectot I on th ♦ ailoceted such os: kinds end 

funnel while decreasing the costs or resource* 
numbers of teachers, space and materials. 

The single most Importent 

I ishment. Accountability req “'^.® |!® V S | uch region Impl les o wl 1 1 Ingness 
accepted by the institution she I learn, Such revls, ^ P ( ^ sub _ syst em 

to change the system after f ndlng causes focusing attention upon 

which does not work. Thls.wi I I be, accompl ished^by r ^9 ^ ^ , flS upon 

organization, personnel, +ecj™> « resul+s g0 ?d educational practice 
+h a c+udent. Without accountability tor resun* 
cannot be Identified end poor practice mey be perpe ue 

,t is believed by meny thet eccountebl I Ity requirements during^ 

the seventies can bring about some urol^egunder which higher educ- 

ment will overcome some ® s P e '{* “ f character I stc^f this cultural leg Is the 
at I on has existed for generations. Character is ^ .. bee(ing up " 

adini ss I on^tondord^or'the curricula* end thus discouraging lerger mm-bers of 
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aspiring learners. For a long time the ratio of college dropout to graduate 
has been a matter of concern and embarrassment to at least some institutions. 
Especially when their examination of those who made it and those who failed 
does not reveal sufficient differences to explain the failure of so many. 

Is it reasonable to believe that the college curriculum assume accountability 
for Its failures as well as the quality of its successes? If it is then 
some performance base must be established and some reliable sensing systems 
introduced to report symptoms of success and failure in progress and before 
the accomplished fact. We are talking about a sensing system analogous to 
the motor temperature gauge on the instrument panel of your car. It tells 
you how hot the engine is getting and gives you an opportunity to take some 
mediating corrective measures before the radiator pressure cap or the soft 
plug in the engine block blow out all of your water. 

At California State College, Long Beach we have been working 
on a design for the study of what we term debilitating stress in the college 
environment which might shed some light upon factors contributing to student 
attrition and survival. We are seeking to identify and describe student 
attributes necessary to perform adequately and to analyze learning tasks 
within the college environment which contribute to student success or failures 
Our design involves an information systems approach and employs human and 
automatic monitors for gathering and feeding stress information into a medi- 
ation center for analysis and feedback. 
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CURRICULUM ACCOUNTABILITY IN HIGHER 
EDUCATION: LESS TIME; FEWER OPTIONS 

Leslie Wilbur 

(Leslie Wilbur is Professor of Higher Education 
an^chairroan of the Department of Higher Educe- 
tion of the University of Southern C aliform . 

He received his B.A. degree from the University 
of Illinois, and his M.A. from the University of 
California (Berkeley). His Ph.D. is from he 
University of Southern California.) 

Ah hfcp outset let me suggest that I am pleased to see an in- 
At the outse , lfc t the development of the curriculum of 
creasing concern for accountability in on the issue must begin with the 
higher education. However, my comments on the issue m 

explication of ®”^^“ n !l^ counCa bility,'' which seems currently to 

t U eed ^frequently and elastically as "motherhood" or 
"freedom" retains its root meaning, to accourt . The co 
tations” 1 of "reckoning," "responsibility " and "liabiUty 

r oi^thr^rretfrf^^i^roL^ff^cu^icui*" »-i*u. 

x’will try to stay with a fairly narrow definition, that of 

3. Moreover, 1 assume that among us there is ~ la “elistance 
foolhardy— *or perhaps so wise—as to offer „ 

Z 7<X a^Tfor chi; 

Compared^^t^the'curricula of%arlier^institutions^that^of 

variety t^anlTthe ^ a ^gJon™f 'taowledge^acCOTpanie^by 

successive year of college. Wit P fessor > s role i n the curriculum 

larger and more complex institutions, teach er's ability to re- 

SfS ftom^efnafs^He^ to approach its apex in higher 

education. nearly a century the American university has held its 

professors accountabte for performance of research and publication, Which 
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have offered progressively larger rewards and thus have climbed steadily on 
the ladder of priorities • On the other handy accountability in curriculum 
development has had relatively few rewards and consequently less attention 
than more visible and lucrative professional activities. 

Nevertheless there is substantial evidence of fundamental 
changes which can increase the likelihood of curriculum change and the de- 
velopment of curriculum accountability. For example, the Carnegie Commis- 
sion's report Less Time : More Options seems to be a visible part of the 
coalescence of attitudes of students, faculty and the general public. A 
general impatience with the years required by college curriculum is an ap- 
propriate stimulus to a closer examination of the what as well as the why 
of learning. 

If higher education moves outside the college walls, and that 
movement seems inexorable, the pressures for the definition of objectives 
and evaluation of achievement in curriculum will grow. Many faculty mem- 
bers must respond to those pressures. Their aggregate response may in- 
crease the feasibility as well as their willingness to be held accountable 
for the curriculum. 

College students are becoming more sophisticated learners, 
and at the same time more impatient with vaguely defined objectives and in- 
appropriate or inadequate evaluation methods. Being held accountable them- 
selves, students are increasing their demands for counterpart requirements 
of the instructor. 

Moreover, the rising costs of higher education, coupled with 
little evidence of increased productivity, contribute to a growing public 
impatience with what seems to be a vague or mystical response to queries 
about the products of the higher learning. Granted that we within the 
system know all the "answers", those stock responses seem to be less accept- 
able even to many of those inside the walls. 

Accountability is also easier to demand when there is a sur- 
plus of labor. For both teaching and administrative faculty members, the 
Seventies will lack the rosy hue of the Fifties and Sixties, when there was 
a short academic labor supply. A surplus labor market should increase the 
tractability of college instructors toward the acceptance of concepts which 
might in earlier decades be more securely rejected without consideration. 

Change in American higher education usually takes place in a 
piecemeal fashion and with general reluctance. The Seventies, perceived 
from the threshold of the Eighties, should reflect substantial change. 

While the academic love for research cannot die, the infatuation should 
weaken appreciably. This decade seems to be one of broad accountability, 
including government, business, and industry. It seems unlikely that any 

of higher education, but especially curriculum, can evade being called 
to account. 
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WHICH WAY, ACCOUNTABILITY? 



Roger Kaufman 

Professor of Human Behavior 
Graduate School of Human Behavior 
United States International University 



Accountability is vary much tilth us these d ays.‘ and it Is so much 
a part of our educational world that some are even talking about drfferen 

kinds of accountability. 

Accountability seems to be unusuaily troublesome ^educators who 

worry about such questions eccomta _ *3^ °^ vial outcom es which are 

legitimate concern that wc might ide y fhni-h K— 12 and 

highly measurable, and predicate eur educational The 

higher education) on achieving some tangible but^illy aocountablllty . 

ESpS^ that can be maie '■measurable" if we are 

only willing to extend the effort. 
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application of the "physics model to ®f“® a ^"’ hatbcraeans and standard 
lot of things which do not lend themselves « arithmetic^m^ ^ ^ ^ auay 

deviations. This latter |toup frequen y^ aotlv i ty . it was to this concern 

with measurement because at is a "bad" measurement that I attempt- 

over the differences between measurement and bad .muvt (Kaufman> EDUCA - 

TIONAL^SYSTEM VlZ 

J^^i^d^r^rST^ssible taxonoa* is reproduced in 
Figure 1. 

scales f or^bo th* se t 1 in^ed ucctional^out comes ^ and for measuring ^ thenb 

TO^dueat^nal^^^cy^ch^w^^chat^hi^^i^st^ed^it^ 0 ^^^*^^^^^^ , 

accountability (Les singer, EVERY KID A wikkuk, „ lt might be 

For instance, if we wanted to U might be silly if 

possible to measure outcomes on an ordina^^ Thus lt mig ht be 

not impossible to measure it w educational- systems to meet needs 

possible to perform purposive desi, in of ed ucationa: ^ not mindlessly 

and to be accountable for achieving . (using the definitions 

attempt to make all goals into measurable objectives Cusing b e- 

implied by the suggested taxonomy) when such is not wltni 
havioral science state of the art. 
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At least one caveat might be noted in the consideration for use of 
the suggested taxonomy of educational outcomes — the validity of the accept- 
ed outcome specification. While it is perfectly possible to prepare measur- 
able objectives, measurability does not mean validity! It is suggested that 
an assessment of educational needs (gaps between current outcomes and desired 
or required outcomes) be conducted to better assure the validity of outcome 
statements and thus assure an accountability which is reasonable and 
functional . 



SUMMARY 

Many outcomes which educators wish to achieve frequently defy 
quantification in terms of interval or ratio scale properties. Rather than 
restrict our accountability efforts to these kinds of outcome statements, a 
possible taxonomy of educational outcomes is suggested for consideration by 
educational planners. Aside from offering a possible standardization of the 
current nomenclature of outcome labels, it offers some possible greater 
flexibility in designing educational programs to achieve a reasonable and 
functional accountability. 



Type of Scale 


Outcome Statement Label 


Nominal 


Goals, 


Ordinal 


Aims , 




Purposes 


Interval 


Objectives (or measurable 


Ratio 


objectives, or performance 
objectives) 



Figure 1. A Possible Taxonomy of Educational Outcomes. 
(Based upon Kaufman, R. EDUCATIONAL SYSTEM PLANNING, Prentice-Hall, 

In press) 
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ACCOUNTABILITY IN THE AFFECTIVE DOMAIN 
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Dr. Mabel C. Purl 

Director of Research and Evaluation 
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Participants : 



Eli M. Bower 

Professor of Educational Psychology 
Director of Joint Doctorate 
University of California, Berkeley 



Newton Metfessel 

University of Southern California 
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Eli M. Bower 



(Eli M. Bower is Professor of Educational Psycho 87 
at the University of California, Berkeley. He re " 

hi B degree from New York University, his 
ST from Coi^bfa 6 Universit y teachers College, and 
his* Ed. D. degree from Stanford University.) 



We have made two serious, almost atio^as^proces^and the school 
tion. The first was the assumption a e or secondary processes 

as its executive institution were aimed at xn thafc or primary 

of thinking. The second error was t enhanced in man divorced and separ- 

processes of thought could e eve p __ s r^g first error was that of 

ated from intellectual or ®®° 0 ^ a ^ r P sensi tive and compassionate younger gen- 
the scholar; the second, that of in t eer allv interwoven with emotional 

eration. Intellectual pr< oce f Z C 1Z±M the^veJely retarded and the psychot- 
thinking in all living humans including the seve y reading a took or 

ic. One can know intellectually what a blind person xs^y ^ can learn 

attending a lecture about visual impair me . at the school for the 

something entirely different ^ £ or par t of a day. Knowing does 

Blind in Berkeley and experie g shuffling around rooms with blinders 

•£= rj sj ■ - -r 

ss or — - intesrate 

emotional and rational thinking. emotional thinking is by far older 

Of the two modes of tbx&x ng, emotional ^ irrational 

and more powerful and mo “ .^““^ng wiS auto^tically occur if children 
reason we assume that emotional , that learning in this modality 

are left alone. In fact it is commonly ^es not iS^Sre with real, i.e. 
is a process of managing ^ ^ t in the way of education-cold 

intellectual, learning. Strong ieerigg ^ chasm below. 

reason may leap over hot products of emotional thinking 

Imagination and creativity are p friend Janos Plesch corn- 

connected to intellectual know ns t e in replied, "There maybe some- 

pared mathematics to fiction writing, and P my methods of thought, I 

thing in what you say. When I examne myself and W to me than my 

come to the conclusion that the gxf t of fantasy has me Times, 

talent for absorbing knowledge" (Clark, R. W. iansue 
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p. 87). Indeed, Hadamard in a study of creative mathematicians found that 
. they thought in creative leaps which they themselves could not explain ratio- 
nally except' to say they "felt" that was the way the solution to a problem 
ought to go. What creative thinkers have as a rule is access to their imagi- 
nation and a way of allowing it to send back signals to the inte ll ect. 

M Freud in one of his few essays on education put it this way, 

"Education can without further hesitation be described as an incitement to 
the conquest of the pleasure principle and to its replacement by the reality 
principle... The substitution of the reality principle for the pleasure prin- 
ciple denotes no dethronement of the pleasure principle but only a safeguard- 
ing of it" (Freud, S. Formulations Regarding the Two Principles in Mental 
Functioning, 1911, p. 18-19). What Freud is saying is that education is a 
stirring up of the mind to control the emotions but such control does not 
denote a putting down of the emotions but an enhancing of it. 

If we were to take this philosophy and concept seriously, what 
would it mean? First, it would mean accepting the legitimacy • of emotional, 
thought and its healthy relationship to development. Emotions are not to be 
the exclusive property of psychiatry, mental, illness or blind aggression. 
Emotions are love, compassion, pity, sympathy and friendship --all that touches 
life with mystery and satisfaction. 

How. does one incorporate this skill with reading, arithmetic, 
science and physical education? As a start we are trying fo ur types of pro- 
grams to help teachers in this endeavor. These are: l) creation and use of 
games and anagames in the classroom, 2) role playing and theatre game tech- 
niques, 3 ) giving teachers interpersonal skills in conducting classroom and 
parent discussions, and 4 ) developing a curriculum for elementary schools to 
help children. learn behavi orally about human behavior. All of these programs 
are aimed at increasing a teacher's instructional skills and repertoire. There 
is no intent to offer solo emotional trips divorced from cognitive learning. 
The games we have in mind are learning experiences much like Scrabble, Mon-* 
opoiy or Democracy (a simulation game of the legislative process). In Ana- 
games we help the teacher develop materials from which children can develop 
their own games. Role playing and theatre games are techniques for helping 
children know more about literature, social studies or current events by 
getting inside the skins of different people and looking at life from a dif- 
ferent vantage point. Role playing is one way of restoring the cognitive- 
affective connection in actual classroom activity. Both games and role' play- 
ing get children, out of their seats, make active participants of them, turn 
teachers into guides and helpers and make teaching exciting and fun. There 
are many opportunities for all to learn in these approaches, including the 
teacher. One cannot get a program such as this into the schools without the 
teachers themselves experiencing these learnings and therefore feeling com- 
fortable and creative in their use. There can be no forced learning or pre- 
scriptive lessons. If gaming or role playing is seen as foolish, unprofitable 
and dull, the class will not use it. If, on the other hand, teachers learn 
the skills well, they will transmit their know-how and enthusiasm to others. 

How are such activities evaluated? A child who has learned math- 
ematics or English can use and be tested on what he has learned. Similarly a 
child who. has learned gaming, role playing or about human behavior can demon- 
strate this learning through his own behavior. Evaluation can be easily 
attained through direct observation and the use of video tapes. 
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